The Hidden Cost of a DNS Cache: How Rust Saved 100 TB

A DNS cache looks like a dictionary with a short memory. A DNS (Domain Name System) resolver receives a name such as example.com, finds its records, and keeps the answer for a while so the next request can avoid an upstream lookup. That feels lightweight until the dictionary contains hundreds of billions of entries.

In a report published August 27, 2026, Cloudflare described how Big Pineapple, the platform behind 1.1.1.1 and other DNS services, cut roughly 100 terabytes from its fleet-wide memory use. The achievement came from five data-layout changes, not a new caching algorithm. At this scale, one wasted byte per entry costs more than 250 gigabytes of RAM. (blog.cloudflare.com)

The cache was optimized for construction

Each cache entry has a key and a value. The key describes the query: its domain name, record type such as A or AAAA, and a few flags. The value holds response records plus metadata such as a timestamp, a hit counter, and a TTL, or Time to Live—the period for which the answer remains usable.

There is also a wrinkle called EDNS Client Subnet (ECS). It allows an authoritative DNS server to vary its answer based on the client’s network, so one query can require several cached versions. More versions make every byte of overhead matter.

Stop paying for mutability

Rust’s Vec<T> is a growable list. It tracks a pointer to its elements, the current length, and its capacity—the number of elements it can hold before another allocation is needed. That is useful while a response is being assembled. It is unnecessary once the response is frozen.

let building = vec![10u8, 20, 30];
let frozen: Box<[u8]> = building.into_boxed_slice;

Box<[T]> is an owned, fixed-length slice stored on the heap. It keeps the data and its length without reserving room for future elements. The same idea applies to String and Box<str>. Cloudflare replaced eight dynamic vector or string fields in each entry, saving 64 bytes per entry before counting unused heap space. That first group of changes was worth more than 15 terabytes across the fleet. (blog.cloudflare.com)

Replace containers with coordinates

The response originally kept the answer, authority, and additional sections in separate lists. Each list needed its own pointer and length. A more compact design stores all records in one contiguous region and remembers where the sections begin:

struct Sections {
 records: Box<[Record]>,
 authority_start: u16,
 additional_start: u16,
}

These offsets are record indexes rather than large memory addresses. A u16 is enough for the section counts involved, so two small integers replace two pairs of pointer-and-length fields. That saves 28 bytes per entry. Rust’s alignment rules add another lesson: fields may need padding so values sit at suitable addresses, and shrinking one field can remove more padding than expected. Packing several booleans into bit flags helped the surrounding structure become smaller still. (blog.cloudflare.com)

Let the query name do double duty

Every DNS record has an owner name, meaning the domain the record belongs to. Most cached records use the same owner as the query, so storing that name beside every record repeats information the lookup already has.

The exception matters. A query for example.com might return a CNAME pointing to cdn.example.com, followed by A records owned by that second name. The compact representation therefore stores no owner for the common case and reconstructs it from the cache key; it stores a full name only when the owner differs. This is different from DNS wire compression. The cache is not adding a pointer to repeated text—it is omitting the text when it can be safely derived.

Do not let a rare record inflate every common one

A Rust enum lets one value hold one of several variants. The enum reserves enough inline space for its largest variant. That is awkward for DNS: an A record needs only an IPv4 address, while less common types may carry several strings and domain names.

In Cloudflare’s representation, the largest NAPTR variant made the full record-data enum 144 bytes. Because A and AAAA records accounted for more than 80% of benchmark traffic, most records carried a large amount of unused space. Boxing the large variants moved their payloads to separate heap allocations and kept common variants smaller, but it also introduced pointer chasing and allocator overhead. A useful intermediate design can still be a poor final design when the cache contains billions of objects.

Keep the bytes ready for the network

The final step stored record data as DNS wire format, the compact binary encoding used in DNS messages, rather than as a list of parsed enum values. The cache kept one byte buffer with a two-byte length before each record:

[length][record bytes][length][record bytes]...

This removes per-record enum padding and replaces many small heap allocations with one contiguous allocation. It also improves memory locality, meaning the processor can find nearby data with fewer slow memory fetches. A, AAAA, TXT, and many DNSSEC records can usually be copied directly into the outgoing response. Records containing names, such as CNAME, NS, MX, and SOA, still need parsing so DNS name compression can be applied.

The design does not store one finished DNS message for every client. Client options can change what belongs in that message, including whether DNS Security Extensions, or DNSSEC, were requested. Keeping reusable record bytes separate preserves that flexibility while avoiding unnecessary parse-and-reserialize work. (blog.cloudflare.com)

The payoff was smaller and faster

The benchmarked footprint fell from 953 bytes to 420 bytes per entry, a 56% reduction. Bytes allocated per entry dropped from about 1.1 kilobytes to 461 bytes. Insert throughput—the number of entries accepted per second—rose from 625,000 to 893,000, while lookup latency fell from 828 nanoseconds to 670 nanoseconds.

Production resident memory, meaning the RAM held by the running process, also fell as releases rolled out between May 18 and July 6, 2026. At the 99th percentile, where 99% of measured instances used less memory, usage declined from 9.3 GB to 5.3 GB. The fleet-wide reduction settled at roughly 100 terabytes, even though resident memory includes more than the cache itself.

The lesson hiding in the cache

How do you optimize a cache without making it slower? Start with the object’s lifecycle. Data that is temporary may need a growable vector; data that becomes immutable may belong in a fixed slice. Information already present in a key does not need to be copied into every value. Data that is usually sent unchanged may be better stored in its final binary shape.

None of these choices is dramatic in a small program. Together, they change the economics of a system serving billions of lookups. At this scale, memory layout is not a footnote beneath the algorithm. It is part of the algorithm’s capacity, latency, and cost.