CDNs

The site feels instant. You built it, you test it, and your laptop is a few network hops from the server it runs on. Then the support queue starts filling with people describing a different product — one where the page frame appears and the fonts, images and JavaScript trickle in behind it. They are not imagining it, and nothing on the server is broken. They are simply far away.

Distance is not a tuning problem. Light in fiber moves at roughly 200,000 kilometers a second, so a signal from Osaka to a server in Virginia — about 11,000 kilometers — takes something like 55 milliseconds each way, and a round trip is double that. Before a single byte of your JavaScript arrives, the browser has spent several of those round trips on the TCP handshake, the TLS handshake and the request itself. That is a floor set by geography, and no amount of faster code lowers it. The only way under it is to stop making the trip.

Copies close to the reader

A city library system does not ship every book from the central archive. It puts copies on the shelves of branch libraries, so a reader walks ten minutes instead of taking a train across the country. The archive still holds the authoritative collection; the branches hold whatever their readers keep asking for.

A content delivery network is that arrangement for your bytes. The CDN operates points of presence — edge servers, in dozens or hundreds of cities — and readers are routed to a nearby one. Your server, now called the origin, stops being the thing everybody talks to and becomes the thing the edges talk to. A reader in Osaka gets the file from an edge in Japan; a reader in Lisbon gets it from an edge in Europe; the origin can sit wherever it likes.

Each edge is a read-through cache with the same two states as any other: the first request for a URL at a given edge is a miss, which the edge itself resolves by fetching from the origin and keeping the answer. Every later request for that URL at that edge is a hit, served from the same city. Because edges are independent, warming one does nothing for the others: a file popular in Europe is still cold in Japan until somebody there asks for it.

What can live at an edge is anything identical for every reader — scripts, stylesheets, fonts, images, video segments, and increasingly whole pages that do not depend on who is logged in. What cannot is anything personalized or transactional, and the rule for deciding is the response header: Cache-Control with a max-age tells browsers and edges how long a copy stays fresh, and s-maxage sets a different, usually longer, life for shared caches like the edge. A response marked private or no-store is telling the edge, correctly, to stay out of it.

Offload is the second win

Latency is the reason people reach for a CDN. Offload is the reason they keep it. Once the edges answer most requests, the origin’s traffic falls to the misses — one fetch per file per edge rather than one per reader — and origin capacity stops tracking your audience size at all. That changes the sizing conversation, and it is also the practical defense against a traffic flood, because the flood lands on a network built for floods.

Put illustrative numbers on it. A release’s new script is requested by a million readers, spread across 200 edges that each keep it once fetched. Without a CDN, the origin sends that file a million times. With one, it sends it about 200 times, one miss per edge, and the edges answer the other 999,800 themselves. That is a five-thousandfold cut, and doubling the audience leaves the origin at about 200: its load now follows the number of edges, not the number of readers.

The edge badges count every request that reached that edge; the origin’s counts only the misses forwarded to it. Watch them separate: by the seventh step both edges are climbing steadily and the origin has moved twice — once for Europe, once for Japan. The first request in each region pays the long trip, and every reader behind it does not.

Invalidation, dodged rather than solved

Caching’s hard problem is knowing when a stored copy stopped being true, and a CDN makes it harder by holding copies in hundreds of places you do not control. Every CDN offers a purge API, and purging is genuinely useful as an escape hatch, but building a deploy process on it means every release depends on a fleet-wide operation completing correctly.

The library has the better answer: a revised book gets a new edition and a new call number, and nobody has to recall the old one. Do the same to your URLs. Include a hash of the file’s contents in its name or query string, and the URL becomes a permanent name for one exact byte string. New bytes produce a new URL, which is cold everywhere by definition, and the old URL is still valid for anybody mid-session. Nothing is ever overwritten, so nothing ever needs invalidating, and the fingerprinted asset can be marked immutable and cached for a year without risk.

That leaves exactly one mutable thing: the HTML document that names the fingerprinted URLs. Keep it on a short leash and the whole scheme holds:

interface Asset {
  path: string;
  contentHash: string;
}

const ONE_YEAR = 31_536_000;

/** A fingerprinted URL is a permanent name for one exact byte string: change
 *  the bytes and the URL changes with them, so nothing needs invalidating. */
function assetUrl(asset: Asset): string {
  return `/${asset.path}?v=${asset.contentHash}`;
}

/** Edge rules really do match on the shape of the URL like this. */
function cacheControl(url: string): string {
  return url.includes('?v=')
    ? `public, max-age=${ONE_YEAR}, immutable`
    : 'public, max-age=0, s-maxage=60, must-revalidate';
}

const app: Asset = { path: 'app.js', contentHash: '9f2c1a' };
const rebuilt: Asset = { path: 'app.js', contentHash: '4b7e12' };
// assetUrl(app)               → /app.js?v=9f2c1a
// assetUrl(rebuilt)           → /app.js?v=4b7e12, so every edge starts cold
// cacheControl(assetUrl(app)) → public, max-age=31536000, immutable
// cacheControl('/orders')     → public, max-age=0, s-maxage=60, must-revalidate

The document’s line reads as three instructions: browsers hold it for no time at all, shared caches may hold it for a minute, and after that everybody asks again. So a deploy is visible worldwide within about a minute, without a purge, and the assets it points at were already at the edge before anybody clicked.

async function fingerprint(path: string, contents: string): Promise<Asset> {
  const bytes = new TextEncoder().encode(contents);
  const digest = await crypto.subtle.digest('SHA-256', bytes);
  const hex = Array.from(new Uint8Array(digest), (byte) =>
    byte.toString(16).padStart(2, '0'),
  ).join('');
  return { path, contentHash: hex.slice(0, 8) };
}

const styles = await fingerprint('styles.css', 'body { margin: 0 }');
const stylesNextDeploy = await fingerprint('styles.css', 'body { margin: 0 }');
const appJs = await fingerprint('app.js', 'render(orders)');
const appJsFixed = await fingerprint('app.js', 'render(orders, user)');

// true: same bytes, same URL
console.log(assetUrl(styles) === assetUrl(stylesNextDeploy));
// false: new bytes, new URL
console.log(assetUrl(appJs) === assetUrl(appJsFixed));