System Design HLD-04.02
L4 vs L7 load balancing
Layer 4 balancers forward connections by IP and port; layer 7 balancers read HTTP to route, terminate TLS and retry. When to use each.
The big idea
Network traffic is described in layers. Near the bottom, packets travel between IP addresses and ports (the transport layer, layer 4). Near the top, those bytes form HTTP requests with paths, headers and cookies (the application layer, layer 7). A load balancer can work at either level, and the choice decides what it can see and therefore what it can do.
A layer 4 load balancerA load balancer that routes TCP or UDP connections using only IP addresses and ports, without reading the application data inside. looks only at addresses and ports. It picks a server for each new connection and then forwards that connection's packets without reading them. That makes it extremely fast and protocol-agnostic. A layer 7 load balancerA load balancer that understands an application protocol such as HTTP, so it can route, modify and retry individual requests based on their content. reads each HTTP request, so it can send /api to one pool and /images to another, terminate TLS, add headers, retry failed requests and cache responses, at the cost of much more work per request.
Think of a building's post room. A layer 4 clerk reads only the building and floor on each envelope and drops it in the right lift: fast, never opens anything. A layer 7 clerk opens each letter, reads what it is about and walks it to the right person: slower, far smarter.
How it works
Step 1: What each layer can see
| Layer 4 | Layer 7 | |
|---|---|---|
| Decides on | source and destination IP and port | host, path, method, headers, cookies |
| Unit of balancing | connection | request |
| TLS | passes it through untouched | usually terminates it |
| Can retry a failed request | no | yes |
| Protocols | any TCP or UDP | the ones it understands (HTTP, gRPC) |
| Cost per byte | very low | much higher |
Step 2: Connection versus request balancing
An L4 balancer picks a server when a connection opens, and every request on that connection goes to the same server. With long-lived connections, such as HTTP/2 or gRPC, that can be a problem: one client opens one connection and sends thousands of requests, all to one server, while others sit idle. An L7 balancer spreads individual requests across servers even when they arrive on a single client connection, which is why gRPC traffic usually needs L7 balancing.
Step 3: Stacking both
Large systems combine them. A fleet of very fast L4 balancers receives all traffic for a public IP and spreads connections across a fleet of L7 proxies; the L7 proxies terminate TLS, inspect requests and route them to the right service:
Try it
Going deeper
Keeping connections alive through changes
An L4 tier must send every packet of a connection to the same backend, even as balancers and backends are added or removed. Simple modulo hashing breaks many connections whenever the server set changes. Production L4 balancers use consistent or rendezvous hashing ( Consistent hashing ) and fallbacks to keep existing connections on their server.
Direct server return
Responses are usually much larger than requests. Some L4 designs let the backend reply directly to the client, bypassing the balancer on the way back, so the balancer only handles the small incoming traffic. It is very efficient but requires network configuration on every backend.
Trade-offs
| Option | Wins when | Loses when |
|---|---|---|
| Layer 4 | Huge throughput, any TCP or UDP protocol, end-to-end encryption, the first tier at the edge | Content-based routing, retries, per-request balancing of long-lived connections |
| Layer 7 | HTTP and gRPC services: path and host routing, TLS termination, retries, observability | CPU cost per request, non-HTTP protocols, true end-to-end encryption |
| L4 in front of L7 | Large sites: cheap scaling of one IP plus smart routing behind it | More moving parts to run and debug |
Common mistakes
- Balancing gRPC with an L4 balancer and finding all requests from a client stuck on one server.
- Expecting an L4 balancer to retry failed requests or route by path.
- Terminating TLS at L7 and forgetting to re-encrypt traffic to backends where required.
- Hashing connections with modulo, so every scale-up breaks many existing connections.
In the wild
GitHub
GLB: GitHub's open source load balancer (opens in a new tab)Theo Julienne's 2018 post describes GitHub's two-tier design. GLB Director is a layer 4 load balancer that "scales a single IP address across a large number of physical machines", sitting in front of layer 7 proxies such as haproxy. Routers spread traffic over directors with ECMP. To keep connections intact when backends change, directors use rendezvous hashing to pick a primary and secondary server for each connection, and a packet that reaches a server without the connection's state gets a "second chance" by being forwarded to the other. The director was rewritten on DPDK, bypassing the Linux kernel to process packets at line rate on commodity hardware.
Check yourself
-
What can a layer 7 load balancer do that a layer 4 one cannot?
Show answer
B. Route requests by URL path or Host header Only a balancer that parses HTTP can see paths, hosts, headers and cookies.
-
Why is an L4 balancer cheaper per byte?
Show answer
B. It forwards packets without parsing or decrypting the application data No TLS termination and no HTTP parsing means very little work per packet.
-
A gRPC client sends 10,000 requests over one HTTP/2 connection through an L4 balancer. Where do they go?
Show answer
B. All to the one server that connection was assigned to L4 balances connections, not requests. Per-request spreading needs L7.
-
Why do large sites put L4 balancers in front of L7 proxies?
Show answer
B. L4 cheaply spreads one public IP across many L7 proxies, which do the smart routing Each tier does what it is best at: cheap packet spreading, then smart request handling.
Recap
- Layer 4 balancers route connections by IP and port without reading the data.
- Layer 7 balancers read HTTP to route by path or host, terminate TLS and retry requests.
- L4 balances connections; L7 balances individual requests, which matters for HTTP/2 and gRPC.
- L4 is cheaper and protocol-agnostic; L7 is smarter and costs more CPU.
- Large systems put L4 in front of L7 proxies to get both.
Next up
- Crack the system design round learning path