System Design HLD-04.04
Reverse proxies (Nginx, Envoy, HAProxy)
Reverse versus forward proxies, and what a proxy in front of your servers does: TLS, compression, caching, buffering, pooling and sidecars.
The big idea
A proxy is a middleman that makes requests on someone else's behalf. The difference between the two kinds is whose side it is on. A forward proxyA proxy that acts for clients: clients send their requests through it to reach other servers, for example a corporate web filter. sits in front of clients and speaks for them to the internet, like a company's outbound web filter. A reverse proxyA proxy that acts for servers: clients connect to it as if it were the server, and it forwards requests to the real servers behind it. sits in front of servers and speaks for them: clients think they are talking to your website, but they are talking to the proxy, which forwards each request to an application server and relays the answer.
That middle position is enormously useful. The proxy can do the generic, repetitive work every web application needs, so application code does not: encryption, compression, caching, protecting slow servers from slow clients, and keeping a pool of warm connections. Nginx, HAProxy and Envoy are the most common examples, and a load balancer is usually a reverse proxy with balancing switched on.
Think of a hotel concierge. Guests never wander into the kitchen; they ask the concierge, who knows which department to ask, packages the answer nicely, and remembers common answers so the kitchen is not bothered twice.
How it works
Step 1: The proxy shields the backend
Step 2: What it does
- TLS termination: certificates and crypto in one place ( TLS and HTTPS ).
- Compression: gzip or Brotli responses, often cutting text payloads by a large factor.
- Caching: serve repeated responses (static files, public pages) without touching the application at all.
- Buffering: a client on a slow phone network may take seconds to upload a request or download a response. The proxy absorbs that slowness, receiving the whole request before forwarding it and accepting the whole response quickly from the application. The application's threads are freed in milliseconds instead of being held for seconds by slow clients, which matters for thread-per-request servers ( Processes, threads and concurrency ).
- Connection pooling: thousands of client connections are funnelled into a small number of long-lived connections to each backend, saving handshakes and backend resources.
- Routing, rate limiting, header rewriting, retries and observability: the layer 7 features from L4 vs L7 load balancing .
Step 3: Sidecars
In a microservice system, the same proxy can run next to every service instance as a sidecarA helper process deployed alongside each instance of a service, typically a proxy that handles its network traffic.. The application talks to its local proxy on localhost, and the proxy handles retries, timeouts, mutual TLS, metrics and tracing for every call in and out. Teams get consistent networking behavior in every language without rewriting libraries. A fleet of sidecars plus a control plane that configures them is a service mesh, covered in Service mesh .
Going deeper
Nginx, HAProxy, Envoy
All three are fast, event-driven proxies (one reason event loops matter). Nginx started as a web server and is widely used for static files and edge proxying. HAProxy focuses on high-performance load balancing at layers 4 and 7. Envoy was designed for dynamic service-oriented systems: its configuration can be changed at runtime through APIs, which is why it underpins many service meshes and cloud load balancers.
Trade-offs
| Option | Wins when | Loses when |
|---|---|---|
| Reverse proxy at the edge | Almost every web service: TLS, compression, buffering and caching in one place | One more hop to run, configure and monitor |
| Sidecar proxy per service | Many services in many languages needing consistent retries, mTLS and telemetry | Resource overhead per instance and operational complexity |
| Doing it in application code | Very small systems with one service and one language | Consistency and reuse as the number of services grows |
Common mistakes
- Exposing application servers directly to the internet, so slow clients and bots tie up application threads.
- Caching personalized responses at the proxy and serving one user's page to another.
- Losing the client's real IP address: proxies must pass it in a header such as
X-Forwarded-For, and applications must trust that header only from the proxy. - Proxy timeouts shorter than the backend's slowest legitimate response, causing mysterious 504 errors.
In the wild
Envoy
What is Envoy (opens in a new tab)Envoy's documentation describes it as an L7 proxy and communication bus for service-oriented architectures, built on the principle that "the network should be transparent to applications". It runs out of process, alongside each application, so it works with any language and can be upgraded across a fleet without touching application code. It has a layer 3 and 4 filter chain for raw TCP and UDP proxying and an HTTP layer 7 filter chain for routing, buffering and rate limiting, first-class HTTP/2 and gRPC support, dynamic configuration through APIs, and built-in health checking, outlier detection, retries, circuit breaking, request shadowing, statistics and distributed tracing.
Check yourself
-
What is the difference between a forward and a reverse proxy?
Show answer
B. A forward proxy acts for clients; a reverse proxy acts for servers Forward proxies sit in front of clients going out; reverse proxies sit in front of servers coming in.
-
How does buffering at the proxy help application servers?
Show answer
B. Slow clients are absorbed by the proxy, so app threads are freed quickly The proxy waits on the slow network; the app sees fast, complete requests and responses.
-
What is a sidecar proxy?
Show answer
B. A proxy running next to each service instance, handling its network traffic Each instance gets a local proxy for retries, timeouts, mTLS and telemetry.
-
Why must applications behind a proxy read X-Forwarded-For carefully?
Show answer
B. It carries the real client IP, but clients could forge it unless only the proxy is trusted to set it Trust the header only when it comes from your own proxy, which overwrites anything the client sent.
Recap
- A reverse proxy stands in front of servers; clients talk to it as if it were the server.
- It terminates TLS, compresses, caches, buffers slow clients and pools backend connections.
- Buffering protects application threads from slow networks.
- Sidecar proxies bring the same features to every service, forming a service mesh.
- Nginx, HAProxy and Envoy are the common choices; Envoy is built for dynamic configuration.
Next up
- Crack the system design round learning path