Netflix is one of the top OTT platform and mostly everyone familiar with it. Netflix handles large categories of movies and television content and user pay the monthly rent to access these content.
In this article we are going to talk about Netflix system design in details. Before starting that lets understand the key requirement to build the Netflix system.
Key requirement:
- Users should able to create account, login and logout using the subscription.
- Users should be able to play video, pause and fast forward.
- Offline download feature and view later
- User should able to change language and view the recommended content and history.
We are going to build the system that should be:
- Highly available
- Scalable
- Low latency
- Secure
Before we move to the high level design lets understand few keywords that we are going to use in this article:
- CDN – Content delivery network is a network of interconnected servers that speeds up webpages for any applications.
- Encoding or Transcoding – This required to converting the video into different formats and resolutions.Example:
Incoming video can be 1080p resolution and bitrate is 3mbps. Transcoding will convert it to multiple format eg 1080p/3mbps, 720p/2mbps, 480p/1mbps, 360p/900kbps, 240p/400kbps.
- ELB – Elastic load balancer is responsible for routing the traffic to front end service. You can read more about Load balancing here.
ELB perform two-tier load balancing where the load is balanced over zones first and then instance using another layer of Load balancer. It use the round robin approach in both tier.
- ZUUL – ZUUL is the gateway service that provides dynamics routing, monitoring, resiliency and security. It provide easy routing based on query parameters URL and path.
- Hystrix – Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries.
- Microservice architecture: Netflix uses Microservices architecture to run all the APIs needed for applications and web apps. Each API call the other microservices for required data and responds with the complete response.
- EVCache (Memcache) – EV Cache is based on memcache used to cache the endpoints and request will be server from cache server instead of original backend server. This will reduced load and improve performance as EV cache is super-fast.
- Apache Kafka and Chukwe – This is used to ingest the data or logs collected in different part of application. It may contains Error logs, UI activities, Performance events, troubleshooting and diagnostic events.
- Apache Spark: Spark used for executing data engineering, data science and machine learning on single node machine or cluster.
Netflix use this for movie recommendations using viewing historical data and on that data it apply machine learning and suggest recommendations.
High Level Design of Netflix:
Netflix works on two clouds AWS and open Connect. These two clouds works together as the backbone of Netflix and both are highly responsible for providing the best video to the users.
The application has mainly 3 components:
- Client – Client can be anything where user browse and use to play Netflix videos. It can be TV, Mobile etc.
- Open connect or Netflix CDN – It’s a CDN service that handle everything related to video streaming from any location without latency.
- Backend – Expect video streaming it take care of everything. This is taken care by AWS cloud.
How it works:
- When any users play the Netflix view using any client. The request will go to ELB – Elastic load balancer.
- ELB will ask ZUUL for the dynamics routing and proxy url. Netty server will work as a proxy server it will send the request and get the response from service.
- Hystrix act as a fault tolerance and stop the cascading failure in case any one backend server is slow. It will also do the Realtime monitoring. This will talk to microservices in the backend.
- Netflix use the microservice architecture to power all of the APIs needed. The microservices are stateless. Along with the Microservices we are using EV Cache to decrease the load from original backend servers and serve faster.
- It will store the billing related information in the MySQL sever. And uses Cassandra to store user viewing related information.
- Netflix also use, apache kafka and chukwe for collecting logs and storing in the elastic search, S3.
- One of the most important feature is Encoding/ transcoding where any new video will go into the process and convert to multiple streams.
Databases:
- MySQL (RDBMS)
- Cassandra (NoSQL)
MySQL Database:
Netflix store data like biling information, user information and transaction information in MySQL because it needs ACIS Compliance. It has Master-Master setup for MySQL for high availability and scalability.
It deployed on AWS EC2 instance using InnoDB.
Cassandra Database:
Cassandra is NoSQL database that can handle large amount go data with high read and write. It provide consistency when the viewing users grows and the user history. It will store all the require details.
Thank you for reading. We hope this gives you a good understanding. Explore our Technology News blogs for more news related to the Technology front. AdvanceDataScience.Com has the latest in what matters in technology daily.
