System Design: Design a Content Delivery Network

A content delivery network places copies of your content on servers around the world so users fetch data from a nearby location instead of crossing the globe. The result is lower latency, reduced origin load, and better resilience to traffic spikes. CDNs like Cloudflare, Akamai, and CloudFront serve trillions of requests per day, and understanding their architecture reveals fundamental lessons about distributed caching.

Functional Requirements

• Serve static content (images, CSS, JavaScript, video segments) from edge locations.
• Cache dynamic content with configurable TTLs and cache keys.
• Support cache purging and invalidation by URL, tag, or prefix.
• Provide HTTPS termination at the edge with custom domain support.
• Support origin failover when the primary origin is unreachable.

Non-Functional Requirements

• Low latency: serve cached content in under 20ms from the nearest edge.
• Global coverage: deploy edge nodes (Points of Presence, or PoPs) on every major continent.
• High availability: survive individual PoP failures by rerouting traffic.
• High throughput: each edge node must handle hundreds of thousands of requests per second.

High-Level Architecture

A CDN has three layers:

1. Edge layer — PoPs distributed globally, each containing a cluster of cache servers.
2. Mid-tier (shield) layer — regional cache clusters that sit between the edge and the origin, absorbing cache misses so the origin sees a fraction of total traffic.
3. Origin layer — the customer’s actual web server or object store.

A request flows: client -> nearest PoP -> (cache hit? serve) -> mid-tier shield -> (cache hit? serve) -> origin.

Routing Users to the Nearest PoP

Two primary techniques get users to the closest edge:

Anycast routing. Advertise the same IP address from every PoP via BGP. The internet’s routing infrastructure naturally sends each user’s packets to the nearest PoP (in terms of network hops). Anycast is simple, requires no client-side logic, and handles failover automatically — if a PoP goes down, BGP withdraws the route and traffic shifts to the next closest PoP.

DNS-based routing. The CDN operates authoritative DNS servers that resolve the CDN domain to the IP of the nearest PoP based on the resolver’s IP geolocation. This gives more control than anycast (you can factor in PoP load, not just network distance) but adds DNS lookup latency and is less precise because the resolver’s location may differ from the user’s location.

Most production CDNs use a combination: anycast for the initial DNS resolution and then anycast or direct IP routing for content delivery.

Edge Cache Design

Each PoP runs a cluster of cache servers. The cache is a large key-value store where the key is derived from the request (URL, headers, query parameters) and the value is the response (headers and body).

Cache key construction. The default key is the full URL, but this must be configurable. For example, a site might want to cache the same page for all users regardless of cookies, or vary the cache by the Accept-Language header. The Vary header and custom cache key rules control this.

Storage. Edge caches use a combination of RAM and SSD. Hot content lives in RAM for sub-millisecond access. Less popular content spills to SSD. Use a two-tier LRU: an in-memory LRU backed by an on-disk LRU. When memory fills, evict to disk rather than discarding.

Cache admission. Not every request should be cached. A cache admission policy (for example, only cache after seeing two requests for the same object within a window) prevents one-hit wonders from evicting popular content. This is especially important for long-tail workloads.

Origin Shielding

Without a mid-tier, every edge PoP independently fetches from the origin on a cache miss. If you have 200 PoPs and a new object is requested simultaneously from all of them, the origin sees 200 requests for the same object. This is called the thundering herd problem.

An origin shield is a designated mid-tier cache that all edge PoPs route their cache misses through. Instead of 200 origin requests, the shield coalesces them into one. The shield itself is a small number of geographically distributed clusters (for example, three regions: US, Europe, Asia).

Request coalescing at the shield works by holding concurrent requests for the same cache key and serving them all from a single origin fetch. This is also called request collapsing.

Cache Invalidation

Cache invalidation is famously one of the hardest problems in computer science. A CDN needs several invalidation mechanisms:

TTL-based expiration. Every cached object has a time-to-live set by the origin’s Cache-Control headers. After the TTL expires, the edge treats the object as stale and either revalidates with the origin (conditional GET with If-None-Match) or fetches a fresh copy.

Purge API. Customers need to force-invalidate content immediately (for example, after publishing a correction). The CDN provides an API to purge by exact URL, by URL prefix, or by surrogate key (a tag attached to related objects). A purge request propagates from the control plane to all PoPs, which delete the matching entries from their caches.

Stale-while-revalidate. To avoid latency spikes when TTLs expire, serve the stale object to the current request while asynchronously fetching a fresh copy in the background. The stale-while-revalidate directive in Cache-Control enables this.

Surrogate keys. Assign tags to cached objects at the origin (via a custom header like Surrogate-Key: product-123 homepage). To invalidate all objects related to product 123, purge by the surrogate key. This is far more practical than tracking individual URLs.

TLS Termination at the Edge

The edge terminates TLS so backend connections can be unencrypted (or re-encrypted with a simpler internal certificate). Each PoP must hold TLS certificates for every customer domain it serves.

SNI-based certificate selection. During the TLS handshake, the client sends the Server Name Indication (SNI) extension. The edge uses this to select the correct certificate from a certificate store. This allows a single IP address to serve thousands of customer domains.

Certificate management. Automate certificate provisioning with ACME (Let’s Encrypt) or a managed CA. Store certificates in a fast key-value store replicated to all PoPs. When a new certificate is issued or renewed, push it to the edge within minutes.

Handling Dynamic Content

Not all CDN traffic is static. Dynamic content (API responses, personalized pages) can also benefit from edge caching, with caveats.

Short TTLs. Cache API responses for seconds or minutes rather than hours. Even a 5-second TTL absorbs traffic spikes and reduces origin load.

Edge compute. Run lightweight logic at the edge (Cloudflare Workers, Lambda@Edge) to personalize cached content. For example, cache the page skeleton and inject the user’s name at the edge without hitting the origin.

Cache segmentation. Use different cache keys for different user segments (logged-in vs. anonymous, country-based pricing). This increases the cache’s working set but allows caching content that varies by a small number of dimensions.

Monitoring and Analytics

• Cache hit ratio: the primary CDN metric. Track it per PoP, per customer, and per content type. A ratio below 80% for static content suggests misconfigured cache keys or TTLs.
• Origin load: requests per second reaching the origin. This should be a small fraction of edge traffic.
• Edge latency: time to first byte from the edge, measured at the client.
• Bandwidth: total bytes served from edge vs. origin, used for capacity planning and billing.
• Purge propagation time: how long it takes for a purge to reach all PoPs.

Scaling Considerations

• Adding PoPs: deploy new PoPs in underserved regions. Anycast routing automatically starts directing nearby users to the new PoP.
• PoP capacity: scale each PoP by adding more cache servers behind an internal load balancer. Use consistent hashing to distribute cache keys across servers so adding a node only invalidates a fraction of the cache.
• Multi-CDN: large customers use multiple CDN providers for redundancy. The origin can switch CDN providers via DNS if one provider has an outage.

Summary

A CDN is a globally distributed, multi-tier cache. The critical design decisions are how you route users to the nearest edge (anycast vs. DNS), how you structure the caching tiers (edge plus shield to protect the origin), and how you handle invalidation (TTLs, purge APIs, and surrogate keys). The origin shield and request coalescing are the most impactful optimizations, turning N concurrent cache misses into a single origin request.