Cloudflare has announced that it has successfully optimized its Pingora proxy, which is written in Rust, to save approximately 100 terabytes of RAM across its global network. This significant memory reduction was achieved through a series of optimizations, primarily focusing on how Pingora handles HTTP headers.
The core of the optimization involved a re-evaluation of how HTTP headers are stored and processed within Pingora. Initially, Pingora would parse incoming HTTP headers and store them in a `HeaderMap` structure, which is a hash map. While functional, this approach proved to be memory-intensive, especially given the high volume of traffic processed by Cloudflare's edge network. The `HeaderMap` would allocate memory for each header key-value pair, leading to considerable overhead.
To address this, Cloudflare engineers implemented a strategy that leverages the fact that many HTTP headers are common and frequently repeated across requests. Instead of storing full copies of these common headers for every request, Pingora now employs a system where it can reference a canonical, pre-allocated copy of these headers. This "deduplication" process significantly reduces the memory footprint by avoiding redundant storage.
Further memory savings were realized by optimizing the parsing process itself. The new approach minimizes temporary allocations during header parsing, ensuring that memory is used more efficiently from the outset. This involved fine-tuning the Rust code to reduce intermediate data structures and directly process header bytes into their final, optimized representation.
The impact of these changes is substantial. By reducing the memory consumption of each Pingora instance, Cloudflare can now handle a greater number of concurrent connections and requests with the same amount of hardware. This not only translates to direct cost savings in terms of RAM but also improves the overall efficiency and performance of their network infrastructure.
Pingora, Cloudflare's custom-built proxy, replaced its previous proxy infrastructure, which included NGINX. The decision to develop Pingora in Rust was driven by the language's strong performance characteristics, memory safety guarantees, and concurrency features, which are critical for a high-performance network proxy. This latest optimization further validates the choice of Rust for such demanding applications.
The 100TB RAM saving represents a cumulative figure across Cloudflare's entire global network of data centers, where Pingora is deployed at the edge to handle incoming web traffic. This optimization is part of Cloudflare's ongoing efforts to enhance the efficiency and scalability of its infrastructure, ensuring it can continue to support the growing demands of the internet.
