System Design HLD-03.06
Real-time: polling, long polling, SSE and WebSockets
Four ways to push updates to clients, what each costs in delay and requests, and how large systems scale millions of persistent connections.
The big idea
HTTP was designed for the client to ask and the server to answer. But chat messages, live scores, notifications, collaborative editing and stock prices all start on the server: something happens, and the client needs to know now. The server cannot simply call the browser, because browsers do not accept incoming connections. So we need a way for the server to push.
There are four classic answers, from simple to powerful: ask repeatedly (polling), ask and let the server wait until it has news (long polling), keep one response open and stream events down it (Server-Sent Events), or open a two-way pipe (WebSockets). The last two deliver updates almost instantly, but they turn every connected user into an open connection the server must hold, which changes how you scale.
Think of waiting for a parcel. You can walk to the door every ten minutes (polling), stand at the door until it comes (long polling), or have the courier phone you when they arrive, over a line that stays open (a persistent connection).
How it works
Step 1: Four delivery patterns
- Short polling: the client sends a request every N seconds. Trivial to build on any HTTP stack, but a message waits on average half the interval, and most requests return nothing.
- Long polling: the client sends a request; the server holds it open until a message arrives or a timeout (say 30 s) passes, then responds, and the client immediately asks again. Delivery is near instant, with ordinary HTTP.
- Server-Sent Events (SSE): the client opens one request, and the server keeps the response open, writing events as they happen in a simple text format. Browsers reconnect automatically and resume from the last event id. It is one-way: server to client.
- WebSockets: an HTTP request is upgraded into a persistent, full-duplex connection. Either side can send messages at any time with tiny framing overhead. Chat, games and collaborative editors use it.
Try it
Try this:
- With the defaults, short polling adds about 5 seconds of delay on average, and most polls are empty.
- Lower the polling interval to 1 second. The delay drops, but the request rate across 100,000 users explodes.
- Raise messages per minute to 60. Long polling now reconnects after every message and starts to look like polling; streaming connections handle it easily.
Step 2: Persistent connections make servers stateful
With SSE or WebSockets, each connected user holds an open connection to one particular server. The application tier stops being stateless ( Stateless vs stateful services ): a message for Alice must reach the exact machine holding Alice's connection. Large systems split the roles:
Practical concerns that come with persistent connections:
- Heartbeats. Load balancers and mobile networks silently drop idle connections. Both sides send small ping messages every 20 to 60 seconds to keep the connection alive and detect dead peers.
- Reconnection. Clients must reconnect with backoff and jitter after any drop, and fetch what they missed, usually by sending the last message id they received.
- Deploys and rebalancing. Restarting a gateway disconnects every client on it. Drain connections gradually, or a deploy triggers a stampede of reconnects.
- Capacity. A tuned server can hold very large numbers of mostly idle connections, since each costs memory but little CPU; the limits are usually memory, file descriptors and how fast messages are fanned out.
Going deeper
Choosing in practice
Use SSE when updates only flow from server to client (notifications, live dashboards, LLM token streaming): it is plain HTTP, works through most proxies and reconnects on its own. Use WebSockets when the client also sends frequent messages (chat, multiplayer, collaborative editing). Keep polling for infrequent checks where a delay of a minute is fine, such as "has my export finished?". Mobile apps add a fourth path: platform push notifications reach the device even when the app is closed.
Ordering and delivery guarantees
A WebSocket delivers messages in order over one connection, but a reconnect can lose messages sent while the client was away. Real systems give each message a sequence number per channel, store recent messages, and let reconnecting clients ask for everything after the last number they saw, the same cursor idea as Pagination patterns .
Trade-offs
| Option | Wins when | Loses when |
|---|---|---|
| Short polling | Rare updates where seconds or minutes of delay are fine; simplest possible | Freshness and efficiency: most requests are empty |
| Long polling | Near-real-time over plain HTTP when streaming is blocked | High message rates, where it degrades into constant reconnects |
| Server-Sent Events | One-way server-to-client streams: notifications, dashboards, token streaming | Frequent client-to-server messages |
| WebSockets | Two-way, high-frequency messaging: chat, games, collaboration | Simplicity: stateful gateways, heartbeats, reconnection and rebalancing |
Common mistakes
- Polling every second from every client and wondering why the API is overloaded.
- Forgetting heartbeats, so connections silently die behind a load balancer's idle timeout.
- Losing messages during reconnects because the client cannot ask for what it missed.
- Routing messages by broadcasting to every gateway instead of only the ones holding subscribers.
In the wild
Slack
Real-time Messaging (opens in a new tab)Sameera Thangu's 2023 post describes how Slack delivers messages. Clients hold WebSocket connections to the nearest regional gateway server, which tracks each user's channel subscriptions. Channel servers, mapped to channels by consistent hashing, hold recent channel state in memory, and when a message is sent it flows from the web app through an admin server to the channel server and then out to every gateway with subscribers, which push it down to clients. The post says the system "delivers messages across the world in 500ms", and that a replacement channel server is ready to serve traffic in under 20 seconds.
Check yourself
-
With 10-second short polling, how long does a message wait on average before the client sees it?
Show answer
B. About 5 seconds A message arrives at a random moment between polls, so it waits half the interval on average.
-
Which option is one-way from server to client over a long-lived HTTP response?
Show answer
B. Server-Sent Events SSE streams events down one open response; the client sends ordinary requests for anything else.
-
Why do WebSocket systems use a pub/sub layer between app servers and gateways?
Show answer
B. So a message reaches whichever gateway holds the recipient's connection Connections live on specific gateways; pub/sub routes each message to the gateways that need it.
-
Why send heartbeat pings on idle WebSockets?
Show answer
B. To keep intermediaries from dropping idle connections and to detect dead peers Load balancers and mobile networks drop idle connections; pings keep them alive and reveal broken ones.
Recap
- Short polling is simple but slow and wasteful; long polling holds requests until there is news.
- SSE streams server-to-client events over HTTP; WebSockets are two-way persistent pipes.
- Persistent connections make servers stateful: route messages via pub/sub to the right gateway.
- Heartbeats, reconnection with resume, and connection draining are part of the design.
- Choose by direction and frequency: polling for rare checks, SSE for one-way, WebSockets for chat.
Next up
- Crack the system design round learning path