CDNs and Edge Caching Explained
How a content delivery network puts copies of your files near users. A simulation of an LRU edge cache shows hit rates from 39% to 76%, and a latency model shows why the hit rate matters most.
A CDN (content delivery network) keeps cached copies of content on edge servers close to users, so most requests are answered nearby and only cache misses travel to the origin server. Interviewers expect you to explain that the cache hit rate drives average latency and origin load, and that personalised responses should not be cached at the edge.
A CDN keeps cached copies of content on servers close to users, so most requests never travel to your origin. Its effect is governed by the cache hit rate. In a simulation with popularity skewed like real traffic, an edge cache holding 1% of the objects reached a 39% hit rate and one holding 20% reached 76%.
- In a simulation of 100,000 requests with popularity proportional to 1/r, an LRU edge cache holding 1% of 10,000 objects reached a 39.2% hit rate.
- In the same simulated CDN workload, an edge cache holding 20% of the objects reached a 76.4% hit rate, showing diminishing returns as the cache grows.
- In a latency model with 20 ms per CDN hit and 200 ms per miss, raising the hit rate from 80% to 90% cuts average latency from 56 ms to 38 ms.
- Users reach the nearest CDN edge either through DNS that returns a nearby address or through anycast, where the same IP is announced from many places.
- Putting a content hash in a filename lets a CDN cache the file for a year, because each new version is published under a new URL and needs no purge.
01The problem a CDN solves
A request to a server on another continent pays the round-trip time of the path, often 150 ms or more, and a page needs several round trips for DNS, TCP, TLS and the data itself. Faster servers do not remove that. Moving the content closer does.
A content delivery network is a set of servers (edge locations, or points of presence) in many cities. Your origin server holds the master copy. Edge servers keep copies and answer users from nearby. The origin is contacted only for content the edge does not have, which also takes load off it during traffic spikes.
02How a request flows
The user's browser looks up your domain. Through DNS (which returns the address of a nearby edge) or anycast routing (the same IP announced from many places, so the network delivers to the closest), the request reaches an edge server.
On a cache hit, the edge returns its stored copy immediately. On a miss, the edge fetches the file from the origin, returns it to the user, and stores it for later requests. The first user in a region pays the slow path and later users get the fast one. Many CDNs add a mid-tier cache between edges and the origin, so a miss in one city can be filled from a regional cache and not from your origin.
03What can be cached
Static assets such as images, scripts, stylesheets, fonts and downloads are the classic case. Video streaming splits video into small segments, and each segment is an ordinary cacheable file, which is why CDNs carry most video traffic. Some responses from APIs can be cached for seconds or minutes if they are the same for every user.
Personal or rapidly changing responses are the hard part. Anything with a login cookie, a cart or a user-specific price should normally bypass the cache or be marked private. Caching it by mistake would show one user's data to another, which is one of the most serious CDN misconfigurations.
04The hit rate is what matters
Web traffic is not uniform. A few objects get most requests and a long tail gets very few. I simulated 100,000 requests over 10,000 objects, with the chance of requesting the object of popularity rank r proportional to 1/r, and an edge cache that keeps the most recently used objects. The hit rates were:
cache holds 100 objects (1%): 39.2% hits
cache holds 500 objects (5%): 58.7% hits
cache holds 1,000 objects (10%): 67.4% hits
cache holds 2,000 objects (20%): 76.4% hitsA small cache already catches a large share of traffic because the popular objects are requested over and over. But the returns shrink: going from 10% to 20% of the objects added only 9 points. Your own numbers depend on your traffic shape, and the 1/r model here is an assumption, not a measurement of any real site.
05Why a few points of hit rate matter
Take a simple model: an edge is 20 ms from the user and the origin is a further 180 ms from the edge. A hit costs 20 ms and a miss costs 200 ms. The average latency is hit x 20 + (1 - hit) x 200:
50% hit rate -> 110 ms average
80% hit rate -> 56 ms
90% hit rate -> 38 msGoing from 80% to 90% nearly cuts the average by a third. The tail matters too, since the slowest requests are all misses. That is why teams work to raise the hit rate by tuning what gets cached, rather than only buying more edge capacity.
06Raising the hit rate: keys, TTLs and invalidation
Cache key: by default the full URL is the key. Tracking parameters make the same image look like different objects. In a quick check, four URLs /img/a.png, /img/a.png?utm=1, ...2 and ...3 made 4 cache entries, and 1 once the query string was ignored for that path. Normalise the key (drop irrelevant parameters) where it is safe.
TTL: the Cache-Control: max-age=... header says how long a copy may be reused. Long TTLs raise the hit rate but risk stale content. A common approach is to put a content hash in the filename (app.3f9c1a.js), cache it for a year, and publish new versions under new names, so there is nothing to invalidate.
Invalidation: for content you must change in place, CDNs offer purge by URL or tag. Purges take effect across the network within seconds to minutes and are often rate-limited, so design for versioned URLs first. stale-while-revalidate lets the edge serve an old copy while refreshing in the background.
07Other benefits and limits
Because traffic passes through the edge, CDNs also terminate TLS close to users, absorb denial-of-service floods, and apply rate limits and web application firewall rules before requests reach your servers. Edge compute features let small functions run at the edge.
Limits: dynamic, per-user responses mostly cannot be cached, so the CDN can only speed up the connection to the origin. Cache invalidation mistakes can serve outdated pages. And one provider is a dependency: when a large CDN has an outage, many sites go down together.
08Common interview questions
Q: How does a CDN reduce latency?
A: It serves content from an edge server near the user, and only goes to the origin on a miss.
Q: What is the cache hit ratio and why does it matter?
A: Hits divided by total requests. Misses pay the full trip to the origin, so a higher ratio lowers average latency and origin load.
Q: How do you update a cached file immediately?
A: Purge it, or better, use versioned filenames so the new version has a new URL.
Q: What should not be cached at the edge?
A: Personalised or authenticated responses, unless the cache key includes what makes them different.
Q: How is a user routed to the nearest edge?
A: DNS that returns a nearby address, or anycast routing.
Q: Push CDN or pull CDN?
A: Push means you upload content to the CDN. Pull means the edge fetches from the origin on the first request, which is the common model.