Cache Stampedes in Node.js: How We Use Request Coalescing, Redis Locks, and Stale-While-Revalidate for Hot Keys

A cache stampede happens when a popular cache key expires and every concurrent request misses at the same moment and recomputes the same value against the database. At Devya we prevent it in layers in our Node.js services: we coalesce identical in-flight loads inside each process, take a short Redis SET NX PX lock so only one instance recomputes, and keep a soft expiry so readers get the stale value while a single worker refreshes it in the background.

The cache-aside code our team used to write looked correct and passed every test: read from Redis, query Postgres on a miss, write the result back with a TTL. It only failed under real concurrency, when an expensive key such as a homepage feed, a pricing table, or a dashboard aggregate expired while many requests were reading it. These are our field notes on the layers that actually stop that failure mode in Node.js and NestJS services, and what each layer costs.

Key takeaways

A cache stampede, also called a thundering herd or dog-piling, is a burst of identical cache misses that all recompute the same value at once, usually right after a hot key expires.

In-process request coalescing, meaning one shared in-flight Promise per cache key, removes duplicate loads inside a single Node.js process in about ten lines of code.

A Redis lock taken with SET key token NX PX ttl limits recomputation to one instance across the fleet. It is an efficiency lock, not a correctness guarantee.

A soft expiry stored inside the value, plus a longer hard TTL on the Redis key, lets readers keep getting the stale value while one worker refreshes it, so expiry never blocks a request.

TTL jitter and probabilistic early expiration spread refreshes over time, so keys written together do not expire together.

Why does the cache-aside pattern cause cache stampedes?

Cache-aside causes stampedes because every reader that sees a miss is responsible for refilling the cache, and nothing tells the other readers that someone is already doing it. Cache-aside, also called lazy loading, is the pattern where the application checks the cache first and, on a miss, loads from the database and writes the result back.

When a hot key such as feed:home expires, every request that arrives during the recompute window, meaning the time the expensive query takes, misses and runs the same query. The extra queries slow the database down, a slower query widens the recompute window, and a wider window lets more requests miss. That feedback loop is how a routine cache expiry turns into a database incident while the cache is technically working.

How do we coalesce duplicate cache loads inside one Node.js process?

Request coalescing means concurrent callers asking for the same key share one in-flight Promise instead of each starting their own load. Go's singleflight package popularized the pattern; in Node.js it is a Map from cache key to Promise. The first caller stores its Promise in the Map, later callers return the stored Promise, and a finally() handler removes the entry when the load settles.

Three details matter. Removing the entry in finally() keeps a rejected promise from being reused forever: every waiter receives the same error once, and the next caller starts a fresh load. The coalescing key must include everything that changes the result, such as tenant ID, locale, and permission scope, or one user's data gets handed to another user's request. And one caller's AbortSignal must never cancel the shared load, because other callers are still waiting on it.

React's cache() function is not a substitute. React cache() and Next.js request memoization deduplicate calls within a single server render, while a stampede is many separate requests arriving at once.

How do we stop several Node.js instances from recomputing the same key?

A short-lived Redis lock limits recomputation to one instance, because Redis SET with the NX option only succeeds for the first caller while the lock key exists. In-process coalescing works per process, so ten replicas behind a load balancer still run ten recomputes without a shared lock. With ioredis, the call is redis.set(lockKey, token, 'PX', ttlMs, 'NX'), which returns OK only for the winner.

The lock value is a random token from crypto.randomUUID(), and the lock is released by a small Lua script that deletes the key only if it still holds that token. That compare-and-delete stops an instance from deleting a lock that already expired and was taken by someone else. We set the lock TTL comfortably above the slowest recompute we expect; if a recompute outlives the lock, the worst case is one extra recompute, which is acceptable for a cache.

We treat this strictly as an efficiency lock. A single-node Redis lock without fencing tokens cannot guarantee mutual exclusion through a failover or a long garbage-collection pause, so we never use it to protect payments or inventory. The lock also leaves one open question, which is what the losing instances return. Polling until the value appears adds latency and Redis load, which is why we pair the lock with stale values.

How do we keep cache expiry from ever blocking a request?

Storing a soft expiry inside the cached value, with a longer hard TTL on the Redis key, lets every reader keep serving the old value while one worker refreshes it. It is the HTTP stale-while-revalidate idea applied to an application cache. Each entry holds the value, a freshUntil timestamp, and computeMs, the time the last recompute took.

On every read, a fresh entry is returned as is. A stale entry is also returned immediately, and a background refresh starts through the per-process coalescer and the Redis lock, so at most one instance runs the loader. Only a cold miss, after a cache flush or for a brand-new key, makes a reader wait; lock losers in that case load directly while still coalesced to one query per process. We started with a hard TTL of ten times the soft TTL and shortened it for keys where very old data would mislead users.

On a long-running NestJS process, a fire-and-forget refresh simply finishes on its own. On Vercel Functions or any serverless runtime, the instance can be suspended once the response is sent, so we pass the refresh promise to waitUntil() from @vercel/functions, or schedule it with after() in Next.js, to keep it alive past the response.

How do we spread cache refreshes out over time?

Probabilistic early expiration and TTL jitter spread refreshes over time, so neither one hot key nor thousands of keys written together refresh at the same instant. Probabilistic early expiration makes each reader decide, with a probability that rises as expiry approaches, to refresh before the value actually expires. The technique comes from the 2015 paper Optimal Probabilistic Cache Stampede Prevention by Vattani, Chierichetti, and Lowenstein, and is usually called XFetch.

XFetch triggers a refresh when the current time minus computeMs times beta times the natural log of a random number is at least the expiry time, with beta set to 1 by default. Because the log of a random number between 0 and 1 is negative, the formula moves the current time forward by a random amount scaled by how long the last recompute took, so slow keys start refreshing earlier and cheap keys refresh close to expiry. XFetch needs no coordination, which makes it useful on a cache without an atomic set-if-absent.

TTL jitter solves a different problem. A cache warm-up script, a deploy that flushes the cache, or a bulk import writes thousands of keys within seconds, and with an identical TTL they all expire within the same second later. Random jitter of about ten percent around the base TTL spreads those expiries out. Jitter does nothing for a single hot key, which is why it complements coalescing and locks instead of replacing them.

How should we cache missing rows and failed loads?

Caching a short-lived not-found marker, called negative caching, stops repeated lookups for nonexistent keys from reaching the database every time. Without it, a crawler or a script requesting random IDs bypasses the cache completely, a failure mode usually called cache penetration. We keep negative entries to a few seconds so newly created rows show up quickly.

Failed loads get the opposite treatment: we never cache the error. When the loader throws during a background refresh, the stale entry stays in Redis until its hard TTL, so readers keep getting the last good value while the database recovers. That is the application-cache equivalent of the HTTP stale-if-error directive.

Which cache stampede protection should a team use?

We use these techniques in layers, because each one covers a different scope:

Request coalescing: the scope is one process; it prevents duplicate concurrent loads in that process; the cost is a Map and careful key design.

Redis SET NX PX lock: the scope is all instances; it prevents parallel recomputes across replicas; the cost is two extra Redis round trips per refresh.

Soft TTL with stale serving: the scope is all readers; it prevents requests blocking on expiry; the cost is that readers may see data up to one refresh old.

Probabilistic early expiration (XFetch): the scope is all readers; it prevents a synchronized refresh at the exact expiry moment; the cost is that some refreshes happen early.

TTL jitter: the scope is many keys; it prevents mass expiry of keys written together; the cost is less predictable expiry times.

Negative caching: the scope is missing keys; it prevents repeated misses for nonexistent rows; the cost is a brief delay before new rows appear.

Before building this by hand, we check what the cache library already does. Bentocache for Node.js documents stampede protection and grace periods for stale values, and wrap()-style helpers in other libraries differ on whether they coalesce concurrent calls. We verify behavior with a concurrency test instead of trusting a README: delete the key, fire a few hundred concurrent reads with Promise.all, and assert with node:test that the loader ran exactly once.

FAQ

Q: What is the difference between a cache stampede and cache penetration?

A: A cache stampede is many concurrent misses for a key that exists in the database and has just expired from the cache. Cache penetration is repeated misses for keys that do not exist in the database at all, so the cache never fills; negative caching fixes penetration.

Q: Is a Redis SET NX lock safe enough for cache recomputation?

A: Yes, for caching. A duplicate recompute only wastes some database work, so an occasional failure of a single-node Redis lock is harmless. Anything where two concurrent writers would corrupt data needs a database transaction or fencing tokens instead.

Q: Do React cache() or Next.js request memoization prevent cache stampedes?

A: No. React cache() and Next.js request memoization deduplicate calls within one render of one request. A stampede is many separate requests, which needs process-level coalescing plus a lock or stale serving across instances.

Q: How long should the hard TTL be compared with the soft TTL?

A: The hard TTL should cover the longest data-source outage a team wants to ride out with stale data. Ten times the soft TTL is a reasonable starting point; shorten it for keys where very old data would mislead users.

Q: Is application-level stampede protection still needed behind a CDN with stale-while-revalidate?

A: Usually yes. CDN stale-while-revalidate protects cacheable HTTP responses at the edge, but personalized responses, service-to-service API calls, and expensive queries inside a request still go through the application cache, where the same stampede happens.