DoorDash Uses Envoy and Valkey for a 1.5M RPS Proxy Cache with 99.99999% Availability
Leela Kumili · rss · 2026-07-20 Reader 🖥️
Distributed Caching & InvalidationInfrastructure EfficiencyModularity & CouplingSoftware Design FundamentalsSRE & Incident Response
DoorDash's Entity Cache shows that moving caching out of individual services and into a transparent, infrastructure-layer proxy (Envoy + Valkey) — with event-driven invalidation, staleness-tolerant failure handling, and stampede-avoidance techniques — can absorb very large request volume at extremely high availability without requiring any application code changes.
practical · expository, with one embedded narrative — A news report (InfoQ relaying a DoorDash engineering-blog post) describing an existing production system and its measured outcomes. It doesn't argue for the pattern so much as narrate it and report DoorDash's own numbers as fact, with one production-outage anecdote functioning as informal proof of the reliability claims.
Structure
| Part | Role |
|---|---|
| Problem framing | Establishes the motivation: frequently-accessed, infrequently-changing data caused redundant service-to-service requests, extra backend load, extra compute cost, and higher tail latency as DoorDash's microservices ecosystem grew. |
| Architecture description (Envoy + Valkey, transparent proxy in the service mesh) | Explains the mechanism: a proxy sits between client and upstream services, intercepting requests centrally so cache behavior is managed without touching application code. |
| DoorDash quote reframing Entity Cache as 'more than a traditional caching layer' | States the article's real thesis in DoorDash's own words: a bundle of reliability and performance features, not just a cache. |
| Request-flow mechanics (hit/miss handling) | Explains normal-path operation. |
| Invalidation and failure handling (Kafka-based event-driven invalidation, dual TTL, Envoy health-based rerouting) | Presents the core design tradeoff: freshness vs. availability, resolved in favor of availability during failures. |
| Outage narrative | A single concrete production incident offered as evidence that the failure-handling design worked as intended. |
| Performance-optimization techniques (custom buffer pools, lock-free single-flight, XFetch probabilistic early refresh) | Explains how the platform sustains 1.5M RPS without excessive overhead or stampede failures. |
| Quantified results (allocation reduction, throughput, P99 latency, hit rate, request reduction, proxy overhead) | Supplies the numeric evidence for the overall claim that the pattern works at scale. |
Key terms
- Entity Cache Distributed Caching & Invalidation
DoorDash's specific, named transparent-proxy caching platform built on Envoy and Valkey. - transparent proxy cache Distributed Caching & Invalidation
A cache implemented as a proxy inside the service mesh that intercepts existing HTTP/gRPC calls, so services get caching without any code change or awareness that caching is happening. - dual TTL Distributed Caching & Invalidation
Two separate expiry values per cached entry: a short one marking data as 'fresh,' and a longer one marking the outer bound past which even stale data won't be served — the gap between them is what's servable during an outage. - single-flight Distributed Caching & Invalidation
A lock-free mechanism ensuring that when multiple concurrent requests miss the cache for the same key, only one upstream request is made and the result is shared, rather than each request independently hitting the upstream. - XFetch Distributed Caching & Invalidation
An algorithm that proactively refreshes hot cache entries some random amount of time before their actual expiry, spreading refresh load over time to avoid many requests simultaneously hitting an expired entry. - 99.99999% availability (seven nines)
The headline reliability figure DoorDash reports for Entity Cache itself.
Propositions
- explicitEntity Cache serves over 1.5 million requests per second across more than 100 endpoints and 50 services, at 99.99999% availability.conf: medium
- explicitEntity Cache is built on Envoy and Valkey and runs as a transparent proxy inside DoorDash's Envoy-based service mesh, intercepting HTTP and gRPC requests before they reach upstream services, with no application code changes required.conf: high
- explicitEntity Cache uses Kafka-based event-driven invalidation that compares update timestamps against cached responses to refresh stale entries, instead of issuing distributed cache-deletion commands.conf: high
- explicitEntity Cache uses dual TTL thresholds so slightly stale responses can be served during outages, and Envoy removes unhealthy cache instances and routes requests directly to upstream services when needed.conf: high
- explicitDuring a multi-hour upstream production outage, Entity Cache continued serving stale-but-valid cached data instead of failing, which DoorDash presents as proof the design 'proved critical.'conf: medium
- explicitdevelopedCustom buffer pools reduce memory allocation overhead, a lock-free single-flight mechanism prevents duplicate upstream work on cache misses, and probabilistic early refresh (XFetch) reduces cache stampedes.conf: highextends ←Cloudflare Cuts 100 TB of Memory from 1.1.1.1 DNS Cache · p4 — Applies the same allocation-overhead-reduction strategy DoorDash used at the proxy layer (custom buffer pools reducing allocation overhead) one level deeper — inside the representation of individual DNS cache entries rather than around the cache as a whole.
- explicitThese optimizations reduced allocation rates by 50-60%, increased per-pod throughput roughly 5x, reduced P99 latency spikes by up to 80%, produced cache hit rates above 90%, cut upstream requests by 60-95% during normal operation, improved latency up to 90% for newly onboarded endpoints, and added approximately 2.1ms of P99 proxy overhead.conf: medium
- implicitCentralizing caching in the infrastructure/proxy layer is implicitly treated as strictly preferable to letting each service implement its own caching logic, at DoorDash's scale.conf: medium
- implicitServing stale cached data during an outage is treated as an acceptable reliability tradeoff for the endpoints Entity Cache fronts — availability is prioritized over strict freshness.conf: medium
- implicitThe 2.1ms of added P99 proxy overhead is implicitly accepted as a worthwhile cost given the throughput and latency wins, without the article weighing it against the added network hop every proxied service-to-service call now takes.conf: low
Arguments
Entity Cache's centralized, infrastructure-layer caching is a superior approach to ad hoc, per-service caching for a large microservices ecosystem. strength: medium
Explicit premises:
- Frequently accessed but infrequently changing data caused repeated requests, increased backend load, extra compute cost, and higher tail latency as DoorDash's microservices ecosystem expanded.
- Entity Cache lets teams adopt centralized caching without implementing separate solutions within individual applications.
- Entity Cache achieves 1.5M RPS at 99.99999% availability, >90% hit rates, and measured reductions in upstream requests, allocation rate, and latency.
Implicit premises:
- Centralized, infrastructure-layer caching scales and generalizes better across many services than distributed, per-team caching implementations would.
- The operational and complexity cost of running one shared caching proxy that many services now depend on is lower than the aggregate cost of many teams building and maintaining separate caches.
- The specific numbers reported generalize from DoorDash's particular workload and traffic shape to caching problems more broadly, rather than being an artifact of DoorDash's specific request patterns.
Inferences:
- From 'reduces backend load, reduces latency, and achieves a high hit rate' to 'this pattern is broadly recommendable for microservices architectures at scale.'
The staleness-tolerant design (dual TTL plus event-driven invalidation plus health-based rerouting) makes Entity Cache reliable enough to keep serving through infrastructure failures. strength: low
Explicit premises:
- Dual TTL thresholds allow 'slightly stale' responses to be served during outages.
- Envoy removes unhealthy cache instances and routes requests directly to upstream services when needed.
- During an actual multi-hour outage, the system kept serving stale-but-valid data instead of failing.
Implicit premises:
- The data cached by Entity Cache is generally tolerant of staleness across its consuming services — no endpoint proxied through it has real-time consistency requirements where serving stale data would itself be a correctness failure (e.g., live pricing, order status, checkout inventory).
- A single observed outage is representative of failure-handling behavior in general, rather than a favorable, cherry-picked anecdote.
Inferences:
- From 'this design worked during one observed outage' to 'this design is validated for production reliability at DoorDash's full scale and across all use cases.'
Narratives
- A multi-hour upstream production outage during which Entity Cache kept serving stale-but-valid cached data instead of failing.
→ Functions as the article's sole proof-by-anecdote for the reliability and failure-handling design — the one piece of 'it worked in production' evidence backing the availability and staleness-tolerance claims.
Questions addressed
- ✅ How can a large microservices architecture reduce redundant service-to-service requests and the backend load/tail latency they cause, without requiring every team to build its own caching logic?
Centralize caching in a transparent proxy (Entity Cache, on Envoy + Valkey) at the service-mesh layer, requiring no application code changes. - ✅ How does a shared cache stay fresh without expensive fan-out cache-invalidation broadcasts across a large distributed system?
Kafka-based event-driven invalidation compares update timestamps to cached-response timestamps, rather than issuing distributed delete commands. - ✅ How does the cache avoid stampede/thundering-herd failures when many popular keys expire simultaneously under massive request volume?
Lock-free single-flight deduplicates concurrent misses for the same key, and probabilistic early refresh (XFetch) refreshes hot entries before they expire. - ✅ What should happen to cached data and its consumers when the caching layer or upstream services experience partial failures — does the system fail closed (error) or fail open (serve something)?
The system fails open: it degrades to serving slightly-stale data within a second TTL threshold rather than failing the request, and Envoy reroutes around unhealthy cache instances.
Gaps
- incompleteNo discussion of which endpoints or data types are eligible for stale-serving during outages versus excluded because staleness would be unsafe (e.g., live pricing, order status, checkout inventory). The failure-handling story treats 'keep serving' as an unqualified win without explaining how DoorDash decides, per endpoint, whether serving stale data is safe.
Read next: Staleness-classification frameworks and per-endpoint consistency tiering in caching architectures; how large-scale caching platforms (Netflix EVCache, Facebook TAO/McRouter) segment cacheable data by consistency requirement. - incompleteNo cost or resource-overhead accounting for running Entity Cache itself — Valkey cluster sizing, compute/memory footprint, or the on-call/operational burden of a new shared critical-path service sitting in front of 100+ endpoints across 50 services. Every efficiency figure reported (allocation rate, throughput, latency) describes downstream savings; none price the caching layer's own resource consumption.
Read next: TCO/cost-accounting treatments of shared infrastructure platforms — what it costs to build and run the platform versus what it saves the consumers of the platform. - incompleteThe 99.99999% (seven-nines) availability figure has no defined measurement window, no statement of whether it's client-observed or server-measured, no indication of whether 'serve stale during outage' counts as a success or a degraded state within that number, and no comparison to the availability of the upstream services being proxied — so it's unclear whether the cache is actually more available than what it fronts, or the number simply describes Entity Cache's own component uptime in isolation.
Read next: SRE literature on error budgets and availability measurement methodology (e.g., Google SRE book) — how availability figures are actually computed and what 'available' means when a system degrades rather than fails. - incompleteNo discussion of blast radius or single-point-of-failure risk from inserting a shared proxy cache in front of 50 services and 100+ endpoints: what happens if Valkey itself fails or is overloaded, whether Entity Cache has its own dependency on Envoy's control plane that could create a correlated failure, and how a bug in the shared invalidation logic would propagate across every service that depends on it — a risk the per-service caching this replaced did not carry, since a bug in one service's cache couldn't affect another's.
Read next: Blast-radius and shared-fate analysis for centralized infrastructure services; postmortems of caching-layer or service-mesh outages that cascaded across many dependent services.closed byPresentation: Understanding Progressive Collapse: How To Avoid A Cascading Failure — Supplies the general mechanism the DoorDash gap asked for — a detailed postmortem-based account of how a shared, centralized infrastructure layer (AWS's DNS-route management, analogous to a shared proxy/cache layer sitting in front of many services) can cascade to every dependent service when it fails, plus the specific paradox that the redundancy added to that shared layer for resiliency was itself the cause. Not caching-specific, but directly on-point for centralized-infrastructure shared-fate risk. - incompleteNo comparison to alternative architectures for the same problem — CDN-style edge caching, a shared client-side/library cache instead of a proxy, a dedicated caching-as-a-service product, or a simpler TTL-only cache without event-driven invalidation — so a reader can't tell whether the added complexity (Kafka pipeline, dual TTL, single-flight, XFetch) is justified relative to simpler designs, or what specifically necessitated the proxy-layer approach over a client-library approach.
Read next: Comparative surveys of caching architecture patterns (proxy/sidecar vs. client library vs. CDN edge vs. managed caching service) and the conditions under which each is the right choice. - illogicalThe performance and availability numbers are exclusively DoorDash's own self-reported figures, relayed by the news article without independent verification, baseline comparison, or stated methodology; ranges like '50-60%' allocation reduction and 'up to 80%' latency reduction are given without specifying which endpoints, what traffic pattern, or what time window they were measured under — the article treats vendor-blog numbers as established fact rather than claims requiring corroboration.
Read next: General critique of vendor engineering-blog metrics as a genre; look for independent benchmarking or a follow-up postmortem/retrospective on Entity Cache's actual production behavior over a longer horizon.
Related documents
- AI Is Changing Software Engineering. The Fundamentals Still Matter.
- 5 Things World-Class Engineers Do That You Don't
- The "funhouse mirror": How AI reflects the hidden truths of your software pipeline
- Article: Implementing Chaos Engineering in Financial Payment Systems: Lessons from Enterprise ECS Deployments
- Cloudflare Cuts 100 TB of Memory from 1.1.1.1 DNS Cache
- Article: Modular Edge Computing at Multi-Tenant SaaS Scale on Cloudflare Workers
- Presentation: Accelerating Performance by Incrementally Integrating Rust Into Existing Codebase
- JEP 540 Proposed to Target JDK 28 with a Simple JSON API
- Worse is better: C# versus F#
- Software Design in the Agentic Age: Place Your Bets