Skip to content
learn
  1. Learn
  2. High Level System Design
  3. APIs and Communication

System Design HLD-03.06

Real-time: polling, long polling, SSE and WebSockets

Four ways to push updates to clients, what each costs in delay and requests, and how large systems scale millions of persistent connections.

Intermediate 14 min Last reviewed

The big idea

HTTP was designed for the client to ask and the server to answer. But chat messages, live scores, notifications, collaborative editing and stock prices all start on the server: something happens, and the client needs to know now. The server cannot simply call the browser, because browsers do not accept incoming connections. So we need a way for the server to push.

There are four classic answers, from simple to powerful: ask repeatedly (polling), ask and let the server wait until it has news (long polling), keep one response open and stream events down it (Server-Sent Events), or open a two-way pipe (WebSockets). The last two deliver updates almost instantly, but they turn every connected user into an open connection the server must hold, which changes how you scale.

Think of waiting for a parcel. You can walk to the door every ten minutes (polling), stand at the door until it comes (long polling), or have the courier phone you when they arrive, over a line that stays open (a persistent connection).

How it works

Step 1: Four delivery patterns

Message delivery with polling, long polling, Server-Sent Events and WebSockets Four lanes between a client and a server. Polling: repeated requests, most come back empty, the message waits for the next poll. Long polling: the request is held open until a message arrives, then the client immediately asks again. Server-Sent Events: one long-lived response streams events from server to client. WebSocket: one persistent two-way connection carrying messages in both directions. Short polling Long polling Server-Sent Events (one-way stream) WebSocket (two-way pipe) any news? no . any news? no . any news? yes any news? ... (held open) ... yes, then ask again event . event . event messages both ways, any time Client Server Client Server Client Server Client Server Figure 1. Polling asks on a timer and most answers are empty. Long polling holds each request until there is news. SSE streams events down one open response. A WebSocket is one open pipe in both directions.
  • Short polling: the client sends a request every N seconds. Trivial to build on any HTTP stack, but a message waits on average half the interval, and most requests return nothing.
  • Long polling: the client sends a request; the server holds it open until a message arrives or a timeout (say 30 s) passes, then responds, and the client immediately asks again. Delivery is near instant, with ordinary HTTP.
  • Server-Sent Events (SSE): the client opens one request, and the server keeps the response open, writing events as they happen in a simple text format. Browsers reconnect automatically and resume from the last event id. It is one-way: server to client.
  • WebSockets: an HTTP request is upgraded into a persistent, full-duplex connection. Either side can send messages at any time with tiny framing overhead. Chat, games and collaborative editors use it.

Try it

This interactive needs JavaScript. The diagrams and text above cover the same idea.

Try this:

  1. With the defaults, short polling adds about 5 seconds of delay on average, and most polls are empty.
  2. Lower the polling interval to 1 second. The delay drops, but the request rate across 100,000 users explodes.
  3. Raise messages per minute to 60. Long polling now reconnects after every message and starts to look like polling; streaming connections handle it easily.

Step 2: Persistent connections make servers stateful

With SSE or WebSockets, each connected user holds an open connection to one particular server. The application tier stops being stateless ( Stateless vs stateful services ): a message for Alice must reach the exact machine holding Alice's connection. Large systems split the roles:

Connection gateways with pub/sub routing Clients hold WebSocket connections to several gateway servers. When a message is sent, the application publishes it to a pub/sub layer keyed by channel or user. Every gateway subscribed to that channel receives it and pushes it down the right connections. publish Alice Bob Chen Gateway 1 holds sockets Gateway 2 holds sockets Pub/sub channel -> gateways App servers stateless Figure 2. Gateways only hold connections. The application publishes each message to a pub/sub layer, and whichever gateways hold the recipients' connections receive it and push it down. Gateways can be added without touching the application.

Practical concerns that come with persistent connections:

  • Heartbeats. Load balancers and mobile networks silently drop idle connections. Both sides send small ping messages every 20 to 60 seconds to keep the connection alive and detect dead peers.
  • Reconnection. Clients must reconnect with backoff and jitter after any drop, and fetch what they missed, usually by sending the last message id they received.
  • Deploys and rebalancing. Restarting a gateway disconnects every client on it. Drain connections gradually, or a deploy triggers a stampede of reconnects.
  • Capacity. A tuned server can hold very large numbers of mostly idle connections, since each costs memory but little CPU; the limits are usually memory, file descriptors and how fast messages are fanned out.

Going deeper

Choosing in practice

Use SSE when updates only flow from server to client (notifications, live dashboards, LLM token streaming): it is plain HTTP, works through most proxies and reconnects on its own. Use WebSockets when the client also sends frequent messages (chat, multiplayer, collaborative editing). Keep polling for infrequent checks where a delay of a minute is fine, such as "has my export finished?". Mobile apps add a fourth path: platform push notifications reach the device even when the app is closed.

Ordering and delivery guarantees

A WebSocket delivers messages in order over one connection, but a reconnect can lose messages sent while the client was away. Real systems give each message a sequence number per channel, store recent messages, and let reconnecting clients ask for everything after the last number they saw, the same cursor idea as Pagination patterns .

Trade-offs

OptionWins whenLoses when
Short polling Rare updates where seconds or minutes of delay are fine; simplest possible Freshness and efficiency: most requests are empty
Long polling Near-real-time over plain HTTP when streaming is blocked High message rates, where it degrades into constant reconnects
Server-Sent Events One-way server-to-client streams: notifications, dashboards, token streaming Frequent client-to-server messages
WebSockets Two-way, high-frequency messaging: chat, games, collaboration Simplicity: stateful gateways, heartbeats, reconnection and rebalancing

Common mistakes

  • Polling every second from every client and wondering why the API is overloaded.
  • Forgetting heartbeats, so connections silently die behind a load balancer's idle timeout.
  • Losing messages during reconnects because the client cannot ask for what it missed.
  • Routing messages by broadcasting to every gateway instead of only the ones holding subscribers.

In the wild

Slack

Real-time Messaging (opens in a new tab)

Sameera Thangu's 2023 post describes how Slack delivers messages. Clients hold WebSocket connections to the nearest regional gateway server, which tracks each user's channel subscriptions. Channel servers, mapped to channels by consistent hashing, hold recent channel state in memory, and when a message is sent it flows from the web app through an admin server to the channel server and then out to every gateway with subscribers, which push it down to clients. The post says the system "delivers messages across the world in 500ms", and that a replacement channel server is ready to serve traffic in under 20 seconds.

Check yourself

  1. With 10-second short polling, how long does a message wait on average before the client sees it?

    Show answer

    B. About 5 seconds A message arrives at a random moment between polls, so it waits half the interval on average.

  2. Which option is one-way from server to client over a long-lived HTTP response?

    Show answer

    B. Server-Sent Events SSE streams events down one open response; the client sends ordinary requests for anything else.

  3. Why do WebSocket systems use a pub/sub layer between app servers and gateways?

    Show answer

    B. So a message reaches whichever gateway holds the recipient's connection Connections live on specific gateways; pub/sub routes each message to the gateways that need it.

  4. Why send heartbeat pings on idle WebSockets?

    Show answer

    B. To keep intermediaries from dropping idle connections and to detect dead peers Load balancers and mobile networks drop idle connections; pings keep them alive and reveal broken ones.

Recap

  • Short polling is simple but slow and wasteful; long polling holds requests until there is news.
  • SSE streams server-to-client events over HTTP; WebSockets are two-way persistent pipes.
  • Persistent connections make servers stateful: route messages via pub/sub to the right gateway.
  • Heartbeats, reconnection with resume, and connection draining are part of the design.
  • Choose by direction and frequency: polling for rare checks, SSE for one-way, WebSockets for chat.

Get new lessons by email

Get new posts on backend and AI internals. No spam, unsubscribe anytime.

Your email is only used to send new posts. Privacy

Next up