Distributed Caching & Invalidation
Architectural patterns for caching data across distributed/microservices systems — where caching lives (client library, sidecar, proxy, CDN edge), how it's invalidated (TTL, event-driven, write-through), and how it avoids stampedes and handles staleness during failures — trading strict freshness for load reduction and availability.
Also known as: entity cache, transparent proxy cache, proxy cache, cache stampede, thundering herd, single-flight, XFetch, dual TTL, event-driven invalidation, stale-while-revalidate
How authors use it
Oldest first (extraction date when no publication date is known).
| Author | Their term | Their meaning |
|---|---|---|
| DoorDash Uses Envoy and Valkey for a 1.5M RPS Proxy Cache with 99.99999% Availability Leela Kumili · 2026-07-20 | Entity Cache | DoorDash's specific, named transparent-proxy caching platform built on Envoy and Valkey. |
| DoorDash Uses Envoy and Valkey for a 1.5M RPS Proxy Cache with 99.99999% Availability Leela Kumili · 2026-07-20 | transparent proxy cache | A cache implemented as a proxy inside the service mesh that intercepts existing HTTP/gRPC calls, so services get caching without any code change or awareness that caching is happening. |
| DoorDash Uses Envoy and Valkey for a 1.5M RPS Proxy Cache with 99.99999% Availability Leela Kumili · 2026-07-20 | dual TTL | Two separate expiry values per cached entry: a short one marking data as 'fresh,' and a longer one marking the outer bound past which even stale data won't be served — the gap between them is what's servable during an outage. |
| DoorDash Uses Envoy and Valkey for a 1.5M RPS Proxy Cache with 99.99999% Availability Leela Kumili · 2026-07-20 | single-flight | A lock-free mechanism ensuring that when multiple concurrent requests miss the cache for the same key, only one upstream request is made and the result is shared, rather than each request independently hitting the upstream. |
| DoorDash Uses Envoy and Valkey for a 1.5M RPS Proxy Cache with 99.99999% Availability Leela Kumili · 2026-07-20 | XFetch | An algorithm that proactively refreshes hot cache entries some random amount of time before their actual expiry, spreading refresh load over time to avoid many requests simultaneously hitting an expired entry. |
Documents (3)
- Cloudflare Cuts 100 TB of Memory from 1.1.1.1 DNS Cache Leela Kumili · 2026-09-23 Reader 🖥️
Cloudflare cut its DNS cache's per-entry memory footprint by 56% and freed ~100 TB of fleet-wide working-set memory, without sacrificing (and while improving) throughput and latency, by collapsing the in-memory representation of cache entries toward DNS wire format instead of a typed, allocation-heavy Rust structure. - Article: Implementing Chaos Engineering in Financial Payment Systems: Lessons from Enterprise ECS Deployments Salim Adedeji · 2026-09-08 source ↗ Reader 🖥️
In regulated, stateful payment systems running on ECS, chaos engineering only works if teams first replace the standard stateless-web playbook's three violated assumptions — clean rollback, definable blast radius, and free production experimentation — with a compliance-integrated, four-stage blast-radius graduation, because it is measured behavior under actual injected faults, not configuration values or architecture review, that reveals where these systems actually break. - DoorDash Uses Envoy and Valkey for a 1.5M RPS Proxy Cache with 99.99999% Availability Leela Kumili · 2026-07-20 Reader 🖥️
DoorDash's Entity Cache shows that moving caching out of individual services and into a transparent, infrastructure-layer proxy (Envoy + Valkey) — with event-driven invalidation, staleness-tolerant failure handling, and stampede-avoidance techniques — can absorb very large request volume at extremely high availability without requiring any application code changes.
Propositions touching this concept (7)
By publication month, oldest first (extraction month when no publication date is known).
2026-07
- explicitEntity Cache serves over 1.5 million requests per second across more than 100 endpoints and 50 services, at 99.99999% availability.
- explicitEntity Cache is built on Envoy and Valkey and runs as a transparent proxy inside DoorDash's Envoy-based service mesh, intercepting HTTP and gRPC requests before they reach upstream services, with no application code changes required.
- explicitEntity Cache uses Kafka-based event-driven invalidation that compares update timestamps against cached responses to refresh stale entries, instead of issuing distributed cache-deletion commands.
- explicitEntity Cache uses dual TTL thresholds so slightly stale responses can be served during outages, and Envoy removes unhealthy cache instances and routes requests directly to upstream services when needed.
- explicitDuring a multi-hour upstream production outage, Entity Cache continued serving stale-but-valid cached data instead of failing, which DoorDash presents as proof the design 'proved critical.'
- explicitdevelopedCustom buffer pools reduce memory allocation overhead, a lock-free single-flight mechanism prevents duplicate upstream work on cache misses, and probabilistic early refresh (XFetch) reduces cache stampedes.
- implicitServing stale cached data during an outage is treated as an acceptable reliability tradeoff for the endpoints Entity Cache fronts — availability is prioritized over strict freshness.
Open gaps (8)
- incompleteNo framework is given for deciding which of the concrete numeric findings (93s failover, 2.4x DB load amplification, 14,000 stuck transactions) are specific to this author's stack (400 TPS, this JVM version, this HikariCP tuning) versus generalizable. A reader on a non-JVM stack, a different database, or Kubernetes instead of ECS cannot tell how much of the 'what surprised us' material would transfer.Article: Implementing Chaos Engineering in Financial Payment Systems: Lessons from Enterprise ECS Deployments — read next: Look for a comparable chaos-engineering-in-fintech account on a different runtime/orchestrator (e.g. Kubernetes-based payment platform) to see which findings recur and which are ECS/JVM-specific.
- incompleteThe compliance/change-management approval process is asserted to 'produce an audit trail that satisfies PCI DSS and SOC 2,' but no example of that documentation, approval turnaround time, or an auditor's actual response is shown. How Stage 3/4 experiments get approved in real time, and what happens when a rollback condition triggers mid-experiment under regulatory scrutiny, is left unaddressed.
- incompleteThe financial cost/benefit of running the chaos-engineering program itself is only asserted qualitatively (six to twelve months to Stage 4; 'the regulatory cost may exceed the learning value'), with no quantification of engineering-hours invested versus incidents prevented or dollars saved — creating an asymmetry, since the cost of NOT doing chaos engineering is made vivid with a precise seven-figure SLA-penalty figure in the opening narrative, but the cost of doing it is never quantified to compare against.
- uninformedThe DNS TTL / caching-layer discovery (60s configured, 93s measured) is presented as a surprising, one-off finding specific to this team's stack, but client-side DNS caching outliving configured TTLs is a well-documented class of problem in distributed-systems practice; the article doesn't cite or connect to that broader body of knowledge, leaving readers with a specific war story rather than the general principle that would let them predict it in a different stack.Article: Implementing Chaos Engineering in Financial Payment Systems: Lessons from Enterprise ECS Deployments — read next: Distributed-systems literature or postmortems on DNS-based service discovery and client-side caching (JVM, gRPC, various HTTP client libraries) outside the ECS/AWS context.
- incompleteNo accounting for the engineering/maintenance cost of moving from typed Rust structures (enums, Vec, String) to a raw contiguous byte buffer parsed manually against DNS wire format. Manual buffer parsing is exactly the kind of code where Rust's typed guarantees (exhaustive matching, borrow-checked ownership) normally do the most work; collapsing to a byte buffer likely trades some of that safety net for hand-written bounds-checking or `unsafe` code, which the article never measures or even mentions.Cloudflare Cuts 100 TB of Memory from 1.1.1.1 DNS Cache — read next: Cloudflare's own linked blog post or any public code/RFC discussing how memory safety is preserved around the byte-buffer representation; also the corpus's own Rust-adoption material on where `unsafe` gets used and audited in production Rust.
- incompleteNo discussion of how cache eviction, TTL expiry, or entry removal interacts with the new representation. Box<[T]>/Box<str> and a contiguous wire-format buffer are naturally fixed-size once allocated; the article doesn't address fragmentation behavior of the allocator managing many differently-sized buffers over the cache's lifetime — a concern that Unbound/PowerDNS's slab-based allocation specifically targets, and which the article's own comparison paragraph raises but doesn't follow through on.Cloudflare Cuts 100 TB of Memory from 1.1.1.1 DNS Cache — read next: Unbound/PowerDNS documentation on slab allocators and cache-size configuration; any Cloudflare writeup on the underlying memory allocator (jemalloc/mimalloc/custom) used for the new buffers, and its fragmentation characteristics under DNS's TTL-driven churn.
- uninformedBig Pineapple is said to store "more than 250 billion DNS cache entries at any given time," but the article never clarifies whether this is a concurrent global fleet-wide figure or a per-region/per-shard figure. Given the per-instance memory numbers reported (9.3 GB → 5.3 GB p99), readers cannot reconcile the two into an actual entries-per-instance count, which would be needed to sanity-check the per-entry footprint math end-to-end.Cloudflare Cuts 100 TB of Memory from 1.1.1.1 DNS Cache — read next: Cloudflare's original linked blog post (blog.cloudflare.com/dns-cache-memory-optimization-1111/) likely has the sharding/fleet-topology math needed to reconcile these figures.
- illogicalThe article gives a single, anonymous, unelaborated Reddit comment the same narrative weight as Cloudflare's own instrumented microbenchmark and seven-week production rollout data — quoting it as a peer counterpoint to the engineer's LinkedIn quote, without noting the wildly mismatched evidentiary standard (no data, no named author, no verification) or attempting to quantify, test, or rebut the scale-dependence claim it makes.Cloudflare Cuts 100 TB of Memory from 1.1.1.1 DNS Cache — read next: None needed to see the flaw — it's internal to the piece's own sourcing choices. A follow-up would be independent benchmarks of similar enum-boxing-avoidance techniques at smaller (non-hyperscaler) request volumes, to actually test the commenter's claim rather than just repeat it.