Pattern card: CDNs

The full lessons, with worked examples: Video streaming: why it buffers, Live cricket scores and Board results. Background: Why apps remember things and the caching card.

What it is

A CDN (content delivery network) is a large set of edge servers in many cities, sometimes inside your internet provider's building. Each keeps copies of content and answers the users near it. Your own origin server is asked only when an edge doesn't have a fresh copy.

• Everyday version: a newspaper printed once and dropped at every local stall, instead of everyone travelling to the press.
• The rule that makes it work: the answer must be the same for everyone who asks. Photos, scripts, video pieces, a results page, a live score. Not your cart, balance or chats.
• What it buys: speed (the trip is across town, not across the country), origin protection (most requests never reach you), and bandwidth no single data centre could send.
• What it costs: copies can be a little old, and taking a copy back from thousands of servers is slow and fiddly.

When to use it

Use it when:

• Files are shared and heavy: images, scripts, fonts, downloads, ready-made PDFs.
• Video: every viewer of an episode asks for the same named pieces.
• Huge crowds want one small answer: a cricket score, a sale countdown, a results page. Even a 1–2 second copy absorbs the crowd.
• A spike is predictable: a release on Friday at 9 PM, results at noon. Copy content to the edges in advance.

Don't use it, or keep parts off it, when:

• The answer is personal: carts, balances, messages, anything after login. Fetch those on a separate, uncached path.
• It must be exact this instant: stock at checkout, seat counts, prices at payment.
• It is rarely asked for twice: a copy nobody re-reads is wasted work. The long tail of rare items stays at the origin.

How it works

1. Find a nearby edge. The user's DNS lookup, or the network itself, points them at an edge close to them and away from any that is overloaded.
2. Look up the copy. The edge looks up the request by its cache key, usually the URL. A fresh copy is a hit: answered in about 20 ms, the origin never knows.
3. On a miss, fetch once. If the copy is missing or past its TTL, the edge fetches it. Request coalescing means only one request goes upstream while everyone else asking for the same thing waits for it.
4. Add a shield. An origin shield, a middle tier, sits between the edges and the origin. Hundreds of edges missing at once become one fetch.
5. Serve old while refreshing. Many CDNs hand out the slightly old copy while one refresh runs in the background, so expiry never makes a user wait.
6. Copies leave in three ways: the TTL runs out, the origin sends a purge (cache invalidation), or the edge evicts rarely used items to make room.

A crowd collapsing into one request:

1. 1

   Fans in PatnaPatna edge

   20,000 ask for the score in one second

2. 2

   Patna edgeOrigin

   Copy is 2 s old: one fetch, others wait

3. 3

   OriginPatna edge

   Update 1147, keep for 2 s

4. 4

   Patna edgeFans in Patna

   Same answer to all 20,000

Variants

• Pull (lazy) CDN. How: edges fetch on the first miss and keep the copy for its TTL. Good for: most sites; nothing to manage up front. Cost: the first user in each city pays the slow trip, and a cold edge at the start of a spike sends a burst to the origin.
• Push and pre-warming. How: copy content to the edges before anyone asks, called pre-warming. Good for: a big release, results day, popular episodes pushed into provider buildings during quiet night hours. Cost: storage on edges for things that may not be watched, and a job to check every edge really holds them.
• Long-lived static files with versioned names. How: put a version or content fingerprint in the file name (app.v42.js) and set a TTL of a year. A change ships as a new name, so nothing needs purging. Good for: scripts, styles, images. Cost: pages must point at the new names, so the HTML itself needs a short TTL.
• Short-TTL dynamic answers ("micro-caching"). How: cache an answer everyone shares for 1–2 seconds. Good for: live scores, sale countdowns, trending lists. Cost: users see data up to a TTL old; you must keep personal data out of the response and the key.
• Video segment delivery. How: video is cut into segments of 2–10 seconds per quality, listed in a manifest; players fetch pieces by name. Good for: on-demand and live streaming. Cost: many small requests; live pieces are made seconds before they're needed, so there's no time to pre-copy them.
• Private content. How: a signed URL that expires lets the CDN serve a file only to someone the origin approved. Good for: paid videos, personal downloads like a marksheet PDF. Cost: links must be short-lived and keyed carefully so one user's link isn't a shared copy.

Numbers to know

• Edge hit: about 20 ms from a server in your city, against 100 ms or more for a distant origin.
• Hit rate maths: at a 95% hit rate the origin sees 5% of requests; at 99%, 1%. Moving from 95% to 99% cuts origin traffic five times.
• Origin load with short TTLs: about edges × (1 / TTL), not users. 500 edges each refreshing once a second is about 500 requests a second, whether 1 lakh or 3 crore fans are watching.
• Crowd sum: 3 crore fans each asking every 2 seconds is about 1.5 crore requests a second, all absorbed at the edge.
• Bandwidth sum: 1 crore viewers at an average of 3 Mbps is 3 crore Mbps, about 30 terabits a second. No single data centre sends that; edges are essential.
• Video: 2–10 second pieces; players keep about 30 seconds buffered; the ladder runs about 0.3 Mbps (240p) to 15 Mbps (4K). A 45-minute 1080p episode is about 1.7 GB.
• Photos sum: six product photos, about 600 KB, for 5 lakh shoppers is about 300 GB the CDN serves instead of the shop.
• Delta saving: one cricket ball's update is a few hundred bytes against 20–30 KB for the full scorecard.

Pitfalls

• Personal data in a shared copy. One user's page, cached under a URL everyone uses, is shown to strangers. Keep logged-in responses uncached, and put everything that changes the answer into the key.
• Too many keys. Adding a timestamp, a tracking parameter or a user ID to the URL makes every request unique, so the hit rate falls to near zero. Strip parameters that don't change the answer; bucket "since" values for delta updates.
• Purging as the update plan. Purges take time to reach thousands of servers and can miss some. Prefer versioned names for files, short TTLs for live data, and purge only for rare corrections (one corrected marksheet).
• Cold edges at the spike. At 12:00 every edge misses together. Pre-warm, use a shield and coalescing, and load-test through the CDN, not just against your servers.
• A long TTL on something that changes. A price or an error page cached for an hour stays wrong for an hour. Set TTLs per type of content, and keep error pages tiny and short-lived.
• Assuming the CDN covers everything. Checkout, login and payment still hit your servers at full force. Size them separately and protect them with a rate limiter.

In an interview

Draw the CDN in front of anything shared: images, video, static pages, live scores. Interviewers listen for whether you check "is this the same for everyone?" before caching it.

• State the shape: "Crores of readers, one writer, the same answer for all", or "large files, long tail".
• Do the sum: bandwidth or requests a second at peak, then what reaches the origin at your hit rate or TTL.
• Name the protection: shield, request coalescing, pre-warming before a known spike.
• Say how things go stale and get fixed: TTLs per content type, versioned names, purge for corrections.
• Split the paths: shared answers through the CDN, personal ones direct.

Follow-up: "A huge release is coming on Friday at 9 PM." Copy it to the edges in advance, check they hold it, and make sure a missing piece causes one upstream request, not one per viewer.

Follow-up: "A cached result was wrong. How do you fix it?" Update the source first, then purge that one item (or publish it under a new name), so the next request fetches the correct copy.

Follow-up: "Why poll a CDN for scores instead of keeping a connection to every phone?" Holding crores of open connections needs a big dedicated fleet; for an update every 30 seconds, a 1–2 second CDN copy is cheaper and simpler. Push is kept for alerts.

Follow-up: "Can you cache the home page if it shows the user's name?" Cache the shared shell on the CDN and fetch the name and other personal bits separately after the page loads.

For experienced engineers

Origin load under request collapsing is roughly (edge count or shield count) × refreshes per second per key, so a shield turns hundreds of edges into a handful of upstream fetches. Use stale-while-revalidate and stale-if-error so expiry and origin failures don't reach users, and vary the cache key only on what truly changes the response.