System Design: Netflix Architecture Deep Dive

Netflix serves over 250 million subscribers across 190+ countries. At peak hours, it consumes roughly 15 percent of global internet bandwidth. What started as a DVD-by-mail service in 1997 became the defining example of how to build a cloud-native, globally distributed streaming platform. The architecture behind it is one of the most studied systems in software engineering.

From DVD Rental to Global Streaming

Netflix launched streaming in 2007 as a small add-on to its DVD business. At the time, the entire application was a single Java monolith running in a co-located data center. In 2008, a major database corruption took the service down for three days. That outage became the catalyst for two transformative decisions: migrate everything to AWS, and break the monolith into microservices.

The migration took nearly seven years. By 2016, Netflix had shut down its last data center. Today, the control plane — everything except the actual video bytes — runs entirely on AWS. The video bytes themselves flow through Netflix’s own CDN, called Open Connect.

Think of it like a restaurant chain. The corporate headquarters (AWS) handles menus, orders, billing, and logistics. But the actual food preparation happens at local kitchens (Open Connect appliances) placed inside your neighborhood, so delivery is fast.

Microservices Architecture

Netflix runs over 1000 microservices in production. Each service owns its own data, deploys independently, and communicates over REST or gRPC. The company processes roughly 2 billion API requests per day through its edge gateway.

The API Gateway: Zuul

Every client request — from your phone, TV, or browser — hits Zuul, Netflix’s edge gateway. Zuul handles authentication, routing, load shedding, and request throttling. It processes tens of thousands of requests per second per instance.

Client Request Flow:
  Phone/TV/Browser
        |
    AWS ELB (Load Balancer)
        |
    Zuul (Edge Gateway)
        |-- Authentication
        |-- Rate Limiting
        |-- Dynamic Routing
        |
    Backend Microservices
        |-- User Service
        |-- Playback Service
        |-- Recommendation Service
        |-- Billing Service
        |-- ... (1000+ services)

Service Communication

Services talk to each other through a combination of synchronous REST calls and asynchronous messaging via Apache Kafka. Netflix built several open-source tools to manage this:

• Eureka — service discovery. Every service registers itself with Eureka on startup. When service A needs to call service B, it asks Eureka for a healthy instance rather than hardcoding an IP address.
• Ribbon — client-side load balancing. Instead of routing through a central load balancer, each service picks which instance to call based on latency and health metrics.
• Hystrix — circuit breaker. If a downstream service starts failing, Hystrix trips a circuit breaker and returns a fallback response instead of cascading the failure.

// Hystrix command pattern (simplified)
public class RecommendationCommand extends HystrixCommand<List<Video>> {
    private final String userId;

    public RecommendationCommand(String userId) {
        super(HystrixCommandGroupKey.Factory.asKey("RecommendationGroup"));
        this.userId = userId;
    }

    @Override
    protected List<Video> run() {
        // Call recommendation service
        return recommendationClient.getPersonalized(userId);
    }

    @Override
    protected List<Video> getFallback() {
        // Return trending videos if recommendation service is down
        return trendingService.getTopVideos();
    }
}

This fallback pattern is critical. When you open Netflix and the recommendation service is slow, you still see a homepage — it just shows trending content instead of personalized picks. The user rarely notices.

Data Layer

Netflix uses a polyglot persistence strategy — different databases for different workloads:

Workload	Database	Why
User profiles, billing	MySQL (on AWS RDS)	ACID transactions for financial data
Viewing history, ratings	Cassandra	Write-heavy, globally distributed
Session data, caching	EVCache (Memcached)	Sub-millisecond reads
Search	Elasticsearch	Full-text search across catalog
Event streaming	Apache Kafka	Billions of events per day
Data warehouse	Apache Spark, Druid	Analytics and reporting

Cassandra is the workhorse. Netflix operates one of the largest Cassandra deployments in the world — thousands of nodes across multiple AWS regions, handling millions of operations per second. They chose Cassandra for its linear scalability: need more throughput? Add more nodes.

EVCache, Netflix’s wrapper around Memcached, serves roughly 30 million requests per second at peak. Almost every read-path service checks EVCache before hitting a database.

Open Connect: Netflix’s Custom CDN

This is where Netflix diverges from most cloud-native companies. Rather than relying entirely on third-party CDNs like Akamai or CloudFront, Netflix built its own content delivery network called Open Connect.

How It Works

Netflix places custom hardware appliances — called Open Connect Appliances (OCAs) — directly inside ISP networks worldwide. There are thousands of these boxes in over 1000 ISP locations across 60+ countries. Each OCA is a server packed with hard drives and SSDs, optimized for one thing: serving video bytes.

Content Delivery Flow:

1. Encoding Pipeline (AWS)
   Title uploaded -> Transcoded into 100+ formats
   -> Stored in S3

2. Off-Peak Distribution
   S3 -> Open Connect backbone -> OCA at your ISP
   (Videos pushed overnight when bandwidth is cheap)

3. Playback
   Your device -> Steering service picks closest OCA
   -> OCA streams video directly
   (Never touches AWS or Netflix's core infrastructure)

The key insight is proactive caching. Netflix knows from viewing patterns which titles are likely to be popular in each region. Before a new season of a hit show drops, the encoded files are already sitting on the OCA inside your ISP’s network. When you hit play, the video bytes travel only from your ISP’s server room to your device — often just a single network hop.

Adaptive Bitrate Streaming

Netflix encodes every title into over 100 different renditions — combinations of resolution, bitrate, and codec. A single movie might produce 1200 encoded files. The playback client uses adaptive bitrate streaming to switch between renditions based on your current bandwidth.

Encoding Profile Example (simplified):
  4K HDR:    16 Mbps  (H.265/HEVC)
  1080p:      5 Mbps  (H.264)
  720p:       3 Mbps  (H.264)
  480p:       1 Mbps  (H.264)
  240p:     0.3 Mbps  (H.264)

Each resolution has multiple bitrate tiers.
Total per title: ~100-1200 encoded files.

If your connection degrades mid-stream, the client drops to a lower bitrate within seconds — you see a brief quality reduction rather than a buffering spinner. Netflix optimizes aggressively for minimizing re-buffer events, because data shows users abandon sessions after a few seconds of buffering.

Recommendation Engine

Netflix estimates that 80 percent of the content people watch is discovered through its recommendation system rather than direct search. The recommendation engine is not a single algorithm — it is a collection of dozens of ML models working together.

How Recommendations Work

The system uses collaborative filtering, content-based filtering, and deep learning in combination:

• Collaborative filtering — “users who watched X also watched Y.” Netflix uses matrix factorization across its entire user-item interaction matrix. With 250M+ users and thousands of titles, this matrix is enormous but sparse.
• Content-based filtering — Netflix tags every title with hundreds of micro-genres and attributes (pacing, mood, plot type, lead character type). These tags come from a combination of human taggers and ML classifiers.
• Deep learning models — neural networks that learn temporal patterns. What you watch on Friday night differs from Tuesday morning. The model learns these patterns.

Recommendation Pipeline (simplified):

1. Candidate Generation
   From 15,000+ titles -> narrow to ~500 candidates
   (Collaborative filtering, content similarity)

2. Ranking
   Score each candidate for this user
   (Deep learning model: watch history + context + title features)

3. Row Assembly
   Group ranked titles into themed rows
   ("Because you watched...", "Trending Now", etc.)

4. Artwork Personalization
   Pick the thumbnail most likely to attract THIS user
   (A/B tested per user segment)

The artwork personalization is particularly clever. For the same movie, Netflix might show you an image featuring the romantic leads if you tend to watch romance, or an image featuring the action scenes if you lean toward thrillers. This single optimization significantly improved click-through rates.

A/B Testing at Scale

Netflix runs hundreds of A/B tests simultaneously. Every change to the recommendation algorithm, UI layout, or even the shade of red on the play button goes through rigorous testing. They built an internal experimentation platform that can partition users into test groups and measure impact on engagement metrics within days.

Chaos Engineering: Breaking Things on Purpose

Netflix pioneered the practice of intentionally injecting failures into production systems. The philosophy is simple: if your system is going to fail — and it will — you want to find the weaknesses before your users do.

The Simian Army

Netflix built a suite of tools collectively called the Simian Army:

• Chaos Monkey — randomly kills production instances during business hours. Forces every service to be resilient to instance failure.
• Chaos Kong — simulates an entire AWS region going offline. Tests whether traffic can failover to another region.
• Latency Monkey — injects artificial delays into network calls. Surfaces timeout bugs and missing circuit breakers.
• Conformity Monkey — finds instances that do not adhere to best practices (no auto-scaling, wrong instance type) and flags them.

Chaos Monkey Execution (simplified):

1. Select random production instance
2. Verify service has multiple instances running
3. Terminate the selected instance
4. Monitor: Does the service recover automatically?
   - Yes -> No action needed
   - No  -> Alert the owning team, they fix their resilience gaps

The cultural impact is significant. Because engineers know Chaos Monkey will kill their instances, they design for failure from the start. Every service must handle instance loss gracefully. There is no option to skip this — Chaos Monkey runs in production during business hours, every weekday.

Handling Peak Traffic

Netflix experiences massive traffic spikes during global events. A new season of a hit show can drive traffic 3-4x above normal levels. The system handles this through several strategies.

Predictive Scaling

Netflix knows its release calendar months in advance. Before a major release, the capacity planning team pre-scales AWS instances and ensures the relevant content is fully cached on OCAs worldwide. They run load tests simulating expected peak traffic.

Graceful Degradation

When the system is under extreme load, non-critical features degrade before critical ones break:

1. First to go: personalized artwork (fall back to default thumbnails)
2. Next: recommendation model switches to a simpler, less compute-heavy version
3. Next: search suggestions become less granular
4. Last resort: new user signups are throttled

The playback path — the ability to actually watch a video — is protected above everything else. Netflix would rather show you a blank homepage than interrupt a stream.

Multi-Region Deployment

Netflix runs active-active across three AWS regions (us-east-1, us-west-2, eu-west-1). If an entire region fails, traffic shifts to the remaining two. Zuul and the steering service handle this automatically. The Chaos Kong exercises validate this failover regularly.

Technology Stack Summary

Frontend:     React (web), native SDKs for 2000+ device types
Backend:      Java (most services), Node.js (API layer), Python (ML/data)
Databases:    Cassandra, MySQL, EVCache (Memcached), Elasticsearch
Messaging:    Apache Kafka (billions of events/day)
CDN:          Open Connect (custom hardware at ISPs)
Cloud:        AWS (all control plane services)
ML/Data:      Apache Spark, Presto, Druid
CI/CD:        Spinnaker (built by Netflix, now open source)
Monitoring:   Atlas (custom time-series database)

Key Takeaways

Netflix’s architecture teaches several principles that apply far beyond streaming:

• Build for failure. Do not hope your systems will stay up — prove they can survive outages by testing in production.
• Separate the data plane from the control plane. Video bytes flow through Open Connect; everything else goes through AWS. This separation lets each layer scale independently.
• Cache aggressively and close to the user. OCAs inside ISP networks eliminate most long-haul bandwidth. EVCache absorbs database load.
• Degrade gracefully. When under pressure, sacrifice nice-to-haves (personalized thumbnails) to protect must-haves (playback).
• Invest in observability. With 1000+ services, you cannot debug by reading logs. Netflix built Atlas, its own time-series monitoring system, to handle millions of metrics per second.

The next time you hit play and a 4K stream starts in under two seconds, remember that behind that simplicity is a system with thousands of microservices, a custom CDN embedded in ISPs worldwide, and a team that intentionally breaks things every day to make sure it all keeps working.