Skip to content
learn
  1. Learn
  2. High Level System Design
  3. Load Balancing and Traffic Management

System Design HLD-04.06

Content Delivery Networks

Edge caching close to users: pull and push CDNs, Cache-Control and cache keys, hit ratio, origin shields, invalidation and versioned files.

Beginner 12 min Last reviewed

The big idea

Most of the bytes on a web page never change between users: images, scripts, stylesheets, fonts, videos. Sending all of them from your one data center to every user worldwide is slow for distant users and expensive for you. A content delivery networkContent Delivery Network: a global network of edge servers that cache content close to users and serve it on behalf of your origin servers. keeps copies of that content on servers in hundreds of cities, so each user is served from one nearby. Your own servers, the originThe server or storage that holds the authoritative copy of your content, which the CDN fetches from on a miss., only handle the misses.

The effect is large on both sides. Users get content from a few milliseconds away instead of a hundred or more. The origin sees a small fraction of the traffic, which cuts cost and lets it survive spikes. And because the edge absorbs traffic, CDNs also soak up floods of malicious requests before they reach you.

Think of a bestselling book. Instead of every reader ordering it from the publisher's warehouse, every local bookshop keeps copies on the shelf, and only reorders from the warehouse when it runs out.

How it works

Step 1: Hit and miss

CDN edge locations serving hits nearby and fetching misses from the origin The origin sits in the center. Edge servers in four cities surround it, close to their users. A user in Sydney gets a cache hit from the Sydney edge along a short glowing path. A user in Sao Paulo causes a miss: the edge fetches the file from the origin along a long path, caches it, and serves later users locally. hit: ~10 ms miss: fetch once Origin your servers Edge: London Edge: Sydney Edge: Sao Paulo Edge: Mumbai User User Figure 1. A hit is served from the nearby edge: short and fast (glowing). A miss travels to the origin once, then the edge keeps a copy for everyone nearby until it expires.

The share of requests served from the edge is the hit ratioThe fraction of requests a cache answers itself, without going to the origin.. It decides almost everything: at 95 percent, the origin handles one request in twenty.

Step 2: Headers decide what is cached

The origin tells caches what to do with response headers:

Cache-Control: public, max-age=31536000, immutable   # versioned asset: cache for a year
Cache-Control: public, max-age=60, stale-while-revalidate=30   # public page: 1 minute
Cache-Control: private, no-store   # personal data: never in a shared cache
ETag: "v7-3f2a"                    # lets caches revalidate cheaply (304 Not Modified)

The cache keyThe parts of a request a cache uses to decide whether two requests ask for the same object, typically the URL plus selected headers. decides what counts as the same object: usually the host and path, sometimes the query string or headers like Accept-Encoding. A careless key (including a random tracking parameter, say) splits one file into thousands of copies and wrecks the hit ratio.

Step 3: Updating content

Two approaches:

  • Versioned file names: app.3f2a9c.js. Each build produces new names, so you can cache files forever; the HTML points to the new names. This is the most reliable method for assets.
  • Invalidation (purge): tell the CDN to drop a URL or a tag everywhere. Needed for content at fixed URLs, but it takes time to propagate and is easy to forget.

Step 4: Pull, push and shields

A pull CDN fetches from the origin on the first miss: zero setup, the common case. A push CDN has you upload content ahead of time, which suits large files such as video that you know will be popular. An origin shieldAn intermediate CDN cache layer that all edge locations fetch through, so the origin sees one request per object instead of one per edge. is a middle tier: edges miss to the shield, and only the shield misses to the origin, so a new file costs the origin one request instead of one per city.

Try it

This interactive needs JavaScript. The diagrams and text above cover the same idea.

Try this:

  1. At 90 percent hits, the origin sees 5,000 of 50,000 requests per second. Raise the hit ratio to 99 percent: origin traffic falls ten times again. The last few percent matter most for the origin.
  2. Look at the latency curve: it is a straight line. Each percent of hit ratio removes the same amount of latency.
  3. Set the origin to 300 ms, as for a distant user. A CDN now saves far more time per hit.

Going deeper

Dynamic content

CDNs also help with responses they cannot cache. Users connect to a nearby edge, which completes the TCP and TLS handshakes locally and forwards the request over a warm, optimized connection to the origin. Shorter handshakes and a better long-haul path speed up even fully personalized pages. Some CDNs also run code at the edge for redirects, A/B tests, authentication checks and simple personalization.

Caching personal content by mistake

The most damaging CDN bug is caching a response that contains one user's data and serving it to others. Mark personal responses private or no-store, never vary only on cookies without thinking, and test what your CDN does with Set-Cookie headers.

Trade-offs

OptionWins whenLoses when
Long TTL with versioned file names Static assets: near-100 percent hit ratio and instant updates on deploy Content that must live at a fixed URL
Short TTL with revalidation Public pages and APIs that change every few minutes Hit ratio and origin load compared with long TTLs
Explicit purges Fixed URLs that occasionally change and must update quickly Reliability: forgotten or slow purges leave stale content

Common mistakes

  • Serving assets from fixed names with long TTLs, so users run old code after a deploy.
  • Including irrelevant query parameters or headers in the cache key.
  • Caching personalized or authenticated responses in a shared cache.
  • Measuring only edge latency and ignoring the origin's miss traffic, which is what actually falls over.

In the wild

Netflix

How Netflix works with ISPs around the globe to deliver a great viewing experience (opens in a new tab)

Netflix's 2016 post describes Open Connect, its own CDN. It places Open Connect Appliances directly inside internet service providers' networks, in close to 1,000 locations at the time, and says that close to 90 percent of its traffic was delivered over direct connections between Open Connect and residential ISPs. Instead of waiting for misses, it copies files to these locations during off-peak hours, a push model suited to large, predictable video files. ISPs benefit too, because they need less capacity to the rest of the internet.

Check yourself

  1. A CDN's hit ratio rises from 90 to 99 percent. How does origin traffic change?

    Show answer

    B. It drops to about a tenth Misses fall from 10 percent to 1 percent of requests: a tenfold reduction at the origin.

  2. What is the most reliable way to update cached JavaScript after a deploy?

    Show answer

    B. Give each build's files new versioned names New names are new cache entries; old ones can safely live forever.

  3. Which header keeps a personal page out of shared caches?

    Show answer

    B. Cache-Control: private, no-store private and no-store forbid shared caches from storing the response.

  4. What does an origin shield do?

    Show answer

    B. Adds a middle cache tier so the origin sees one miss per object instead of one per edge Edges miss to the shield; only the shield goes to the origin.

Recap

  • A CDN caches content on edge servers near users and fetches misses from your origin.
  • The hit ratio decides latency and origin load; the last few percent matter most for the origin.
  • Cache-Control headers and the cache key control what is cached and shared.
  • Use versioned file names for assets; purge only when you must.
  • Origin shields, push for large files, and edge connections for dynamic content round it out.

Get new lessons by email

Get new posts on backend and AI internals. No spam, unsubscribe anytime.

Your email is only used to send new posts. Privacy

Next up