Cloudflare Cuts 100 TB of Memory from 1.1.1.1 DNS Cache
Leela Kumili · rss · 2026-09-23 Reader 🖥️
Cloudflare cut its DNS cache's per-entry memory footprint by 56% and freed ~100 TB of fleet-wide working-set memory, without sacrificing (and while improving) throughput and latency, by collapsing the in-memory representation of cache entries toward DNS wire format instead of a typed, allocation-heavy Rust structure.
theoretical · expository — News reporting on an engineering result: the author explains what Cloudflare changed and why it worked (understanding), not arguing that the reader should take a specific action. The implicit practical payload — 'represent cache data closer to wire format, avoid boxing enum variants' — is carried by the reporting rather than argued for directly.
Structure
| Part | Role |
|---|---|
| Opening headline results (footprint -56%, ~100 TB freed, throughput +43%, latency -19%) | conclusion / thesis statement |
| Scale context (250B+ entries) and Sebastiaan Neuteboom's LinkedIn quote | context / human-scale framing |
| The five successive Rust representation changes (Box<[T]>/Box<str>, combined record lists, bitflags, owner-name omission, enum-boxing failure) | evidence / mechanism |
| Reddit commenter's scale-dependence caveat | counter-context / qualification |
| Final design description (contiguous byte buffer in DNS wire format) | conclusion of the mechanism narrative |
| Comparison to Unbound and PowerDNS Recursor cache architectures | context / differentiation |
| Production rollout data (May 18–July 6 2026, p99/p90 memory) and benchmark numbers (per-entry footprint, allocations) | evidence / validation |
| Closing statement on reinvesting freed memory into cache capacity | conclusion / implication |
Key terms
- Big Pineapple
Cloudflare's internal name for its DNS platform/resolver software that backs the 1.1.1.1 public resolver and stores 250B+ cache entries fleet-wide. - per-entry footprint
The memory cost, in bytes, of a single DNS cache entry as measured in Cloudflare's benchmarks (953 bytes before, 420 bytes after) — the primary efficiency metric the article tracks across both microbenchmark and production numbers. - working-set memory
The resident memory actively touched by running DNS cache processes across Cloudflare's fleet — what the ~100 TB freed refers to, as distinct from cache capacity or disk/storage. - wire format
DNS's on-the-wire binary encoding, which Cloudflare's final design uses directly as the in-memory cache storage representation (a contiguous byte buffer), collapsing what would otherwise be two representations (in-memory typed structure, on-the-wire bytes) into one. - cache/memory locality
How close together in memory related pieces of a cache entry sit, and how that affects CPU cache-hit behavior; used to explain why boxing larger Rust enum variants — despite shrinking stack size — was a net loss (separate heap allocations reduce locality) and why the final contiguous-buffer design is a net win.
Propositions
- explicitCloudflare reduced the benchmarked per-entry memory footprint of its DNS cache by 56% (953 bytes to 420 bytes) and freed roughly 100 TB of fleet-wide working-set memory.conf: high
- explicitThe same set of changes increased cache insertion throughput by 43% and reduced lookup latency by 19% — a simultaneous space and speed win, not a classic space/time tradeoff.conf: high
- explicitReplacing Vec and String with Box<[T]> and Box<str> for data that becomes fixed after insertion saved 64 bytes per entry and more than 15 TB fleetwide.conf: high
- explicitCombining answer, authority, and additional records into a single list with compact offsets, packing Booleans into bitflags, and omitting owner names that match the queried domain (reconstructing them from the cache key) further reduced per-entry memory.conf: highextendsDoorDash Uses Envoy and Valkey for a 1.5M RPS Proxy Cache with 99.99999% Availability · p6 — Applies the same allocation-overhead-reduction strategy DoorDash used at the proxy layer (custom buffer pools reducing allocation overhead) one level deeper — inside the representation of individual DNS cache entries rather than around the cache as a whole.
- explicitBoxing larger Rust enum variants — the intuitive move to shrink stack size — was ultimately a net loss, because the separate heap allocations it introduced added overhead and reduced memory locality.conf: highextendsWhy Rust is different, with Alice Ryhl · p4 — Cloudflare's concrete enum-boxing/locality tradeoff is a production instance of the data-structure-design difficulty Alice Ryhl describes as Rust's real learning curve (the borrow checker and allocation model pushing problems back into struct/enum design).
- explicitThe final design stores record data in a contiguous byte buffer using DNS wire format, removing enum and per-record allocation overhead and improving locality; frequently used record types can be copied directly into responses, while records containing domain names still require parsing for DNS name compression.conf: highsupportsWhy Rust is different, with Alice Ryhl · p10 — Cloudflare's measured throughput/latency/footprint wins are concrete evidence for Ryhl's claim that Rust now occupies a 'reliable AND performant' category previously unavailable as a single language choice.
- explicitIn production, rolled out between May 18 and July 6, 2026, p99 resident memory per instance fell from 9.3 GB to 5.3 GB, and p90 memory fell from 6.5 GB to 3.8 GB.conf: high
- explicitCloudflare plans to reinvest the freed fleet-wide memory into increased cache capacity rather than simply banking it as reduced overall memory consumption.conf: high
- explicitCloudflare's approach differs structurally from other recursive DNS resolvers: Unbound manages memory via separate message/RRset/key/negative caches with configurable sizes and slab settings, and PowerDNS Recursor via multiple cache types (packet, record); Cloudflare instead reduces representation and allocation overhead within individual cache entries rather than partitioning or configuring cache regions.conf: high
- implicitMemory-representation tricks that avoid enum boxing (accepting more inline/contiguous storage over typed indirection) only pay off once a system operates at very high request volume; at smaller scale, the same indirection can help rather than hurt.conf: medium
- implicitCollapsing an in-memory cache representation toward the format it will eventually be serialized to (DNS wire format) is treated as generally superior to maintaining a separate, more 'idiomatic' typed in-memory structure for a high-volume, allocation-sensitive cache — because it removes both the parse/format round-trip and much of the allocation surface at once.conf: medium
- implicitThe article treats the microbenchmark per-entry numbers (953 → 420 bytes, 1.1 KB → 461 bytes allocations) and the fleet-wide production numbers (~100 TB freed, p99/p90 memory drops) as mutually corroborating, moving between them without flagging that benchmark-to-production generalization is often unreliable.conf: medium
Arguments
Storing DNS cache records as a contiguous byte buffer in (near-)wire format, rather than as parsed/typed Rust structures with per-field allocations, is the superior in-memory representation for a high-volume DNS cache. strength: high
Explicit premises:
- Boxing enum variants added allocation overhead and reduced memory locality.
- The wire-format buffer design removes enum and per-record allocation overhead and improves locality.
- Frequently used record types can be copied directly into responses from the buffer.
- Benchmarked per-entry footprint fell 56% and production p99/p90 memory fell substantially after adopting this design.
Implicit premises:
- CPU cache locality and allocation count dominate the cost model at Cloudflare's request volume more than the convenience of a typed, idiomatic in-memory structure.
- The parsing cost paid when a record actually needs decomposition (name compression) is rare or cheap enough not to offset the storage savings.
- DNS wire format retains enough information to serve as an internal representation without losing data needed elsewhere in the resolver pipeline.
Inferences:
- If wire-format-as-storage is a generally superior strategy for read-heavy, allocation-sensitive caches, similar representation-collapsing techniques could apply to other network-protocol caches (e.g. HTTP header caches, gRPC message caches) — the article never generalizes beyond DNS to test this.
The specific memory-representation techniques Cloudflare applied — especially avoiding boxed enum variants in favor of inline/contiguous storage — are not universally good practice; they are contingent on operating at very high request volume. strength: low
Explicit premises:
- "A lot of these memory tricks only pay off once you're at Cloudflare's request volume; at a smaller scale the extra indirection from boxing variants can actually hurt cache locality more than it helps." (Reddit commenter)
Implicit premises:
- The tradeoff between allocation overhead and access-pattern-driven locality benefits is a function of request volume, working-set size, or cache size.
- Below some threshold, unboxed inline representations may not repay the fixed complexity/engineering cost of implementing them.
Inferences:
- The article never identifies where that scale threshold approximately sits, or which proxy metric (QPS, entries per instance, cache size) determines which side of the tradeoff a given system falls on — so the caveat, while directionally plausible, gives readers no actionable rule.
Narratives
- The five successive representation changes told as a sequence: Vec/String → Box<[T]>/Box<str>; combined record lists with compact offsets; bitflags for Booleans; owner-name omission reconstructed from the cache key; and the enum-boxing attempt that had to be abandoned before arriving at the final contiguous-buffer design.
→ Implicitly supports the proposition that large aggregate memory wins at scale come from compounding many small, individually modest representation changes — including a documented dead end (enum boxing) — rather than from one big rewrite. - Sebastiaan Neuteboom's LinkedIn quote: "It's not every day you get to save 100 terabytes of memory."
→ Provides human-scale, informal validation of an otherwise abstract fleet-wide number, functioning as social proof that the result is genuinely remarkable within the engineering team itself. - The Reddit commenter's counter-anecdote about indirection only paying off at Cloudflare's scale.
→ Functions as an implicit cautionary narrative — 'engineering wisdom from a hyperscaler doesn't transfer without checking your own scale' — supporting a gap/caveat rather than the article's main thesis, and left unresolved.
Questions addressed
- ✅ Can a DNS cache's per-entry memory footprint be reduced substantially without sacrificing throughput or latency?
Yes — footprint fell 56%, while insertion throughput rose 43% and lookup latency fell 19% simultaneously, contradicting a simple space/time tradeoff expectation. - ✅ Where should DNS record data live in memory — as typed Rust structures (enums, Vec, String) or as a raw wire-format byte buffer?
The wire-format contiguous buffer wins for this workload, arrived at only after typed representations — including boxed enums — proved worse on allocation overhead and locality. - ⭕ At what scale, request volume, or cache size does this representation strategy stop paying off, or start hurting performance instead?
- ⭕ What does the wire-format-as-storage design cost in code complexity, maintainability, or safety, relative to the typed Rust structures it replaced?
Gaps
- incompleteNo accounting for the engineering/maintenance cost of moving from typed Rust structures (enums, Vec, String) to a raw contiguous byte buffer parsed manually against DNS wire format. Manual buffer parsing is exactly the kind of code where Rust's typed guarantees (exhaustive matching, borrow-checked ownership) normally do the most work; collapsing to a byte buffer likely trades some of that safety net for hand-written bounds-checking or `unsafe` code, which the article never measures or even mentions.
Read next: Cloudflare's own linked blog post or any public code/RFC discussing how memory safety is preserved around the byte-buffer representation; also the corpus's own Rust-adoption material on where `unsafe` gets used and audited in production Rust. - incompleteThe article never states at what scale, QPS, or cache size the representation strategy stops paying off, despite including a caveat (the Reddit commenter) that explicitly raises scale-dependence. This leaves practitioners at smaller companies unable to judge whether these specific lessons (avoid boxing enum variants, prefer inline storage) apply to their own systems.
Read next: Benchmarking write-ups from mid-size companies applying similar cache-representation techniques, to help locate the approximate crossover point between Cloudflare's regime and smaller deployments. - incompleteNo discussion of how cache eviction, TTL expiry, or entry removal interacts with the new representation. Box<[T]>/Box<str> and a contiguous wire-format buffer are naturally fixed-size once allocated; the article doesn't address fragmentation behavior of the allocator managing many differently-sized buffers over the cache's lifetime — a concern that Unbound/PowerDNS's slab-based allocation specifically targets, and which the article's own comparison paragraph raises but doesn't follow through on.
Read next: Unbound/PowerDNS documentation on slab allocators and cache-size configuration; any Cloudflare writeup on the underlying memory allocator (jemalloc/mimalloc/custom) used for the new buffers, and its fragmentation characteristics under DNS's TTL-driven churn. - uninformedBig Pineapple is said to store "more than 250 billion DNS cache entries at any given time," but the article never clarifies whether this is a concurrent global fleet-wide figure or a per-region/per-shard figure. Given the per-instance memory numbers reported (9.3 GB → 5.3 GB p99), readers cannot reconcile the two into an actual entries-per-instance count, which would be needed to sanity-check the per-entry footprint math end-to-end.
Read next: Cloudflare's original linked blog post (blog.cloudflare.com/dns-cache-memory-optimization-1111/) likely has the sharding/fleet-topology math needed to reconcile these figures. - illogicalThe article gives a single, anonymous, unelaborated Reddit comment the same narrative weight as Cloudflare's own instrumented microbenchmark and seven-week production rollout data — quoting it as a peer counterpoint to the engineer's LinkedIn quote, without noting the wildly mismatched evidentiary standard (no data, no named author, no verification) or attempting to quantify, test, or rebut the scale-dependence claim it makes.
Read next: None needed to see the flaw — it's internal to the piece's own sourcing choices. A follow-up would be independent benchmarks of similar enum-boxing-avoidance techniques at smaller (non-hyperscaler) request volumes, to actually test the commenter's claim rather than just repeat it.