System Design HLD-02.06
Stateless vs stateful services
What state is, why servers that remember users break when traffic moves, and where to put sessions so any server can handle any request.
The big idea
StateData a system remembers between requests, such as who is logged in, what is in a cart or how far a long upload has got. is anything a system remembers from one request to the next: that you are logged in, what is in your cart, which step of checkout you reached. Every useful product has state. The design question is where it lives.
If a server keeps state in its own memory, the user must keep coming back to that exact server. The moment that server restarts, is replaced or the load balancer sends the user elsewhere, the state is gone: the user is logged out or the cart is empty. A statelessA service that keeps no client data in its own memory between requests, so any instance can handle any request. service instead keeps nothing between requests and stores state in a shared place every server can reach. Then servers become interchangeable, which is exactly what horizontal scaling and failover need.
Picture a coffee shop with several baristas. If your order is only in one barista's head, you must wait for that barista. If it is written on a ticket on the shared rail, any barista can make it, and nothing is lost when one goes on break.
How it works
Step 1: The stateful failure
Step 2: Three places to keep sessions
Sticky sessions. The load balancer pins each user to one server, using a cookie or the client's IP. It is a quick fix for an existing stateful app, but load spreads unevenly, and when the server dies its users still lose their sessions.
A central session store. Servers keep nothing locally and read the session from a fast shared store, usually an in-memory database like Redis, on every request. Any server can serve anyone. The store becomes a critical dependency, so it needs replication, but it is a single, well-understood piece.
Client-side tokens. The server puts the session data, such as user ID and expiry, into a signed tokenA piece of data signed by the server with a secret key, so the server can later verify it was not altered. JWTs are a common format. that the client sends back with each request. Servers verify the signature and need no lookup at all. The catch is revocation: a token stays valid until it expires unless you add a block list, so tokens are usually short-lived. See OAuth 2.0, OpenID Connect and JWT .
Step 3: State still has to live somewhere
Statelessness does not remove state, it moves it to systems built for it:
- Databases for durable data: accounts, orders, posts.
- Caches and session stores for fast, short-lived data.
- Object storage for files and media.
- Queues for work in progress.
The application tier becomes a pool of interchangeable workers that you can add, remove, restart and deploy freely, while the hard problems of keeping state correct are concentrated in a few specialized systems.
Try it
Going deeper
Some services are stateful on purpose
Databases, caches, message brokers and real-time systems like chat servers holding open WebSocket connections are inherently stateful. They are scaled with techniques of their own: replication, partitioning and consistent routing of each user or key to the right node. You will meet these from Replication: leader-follower onward. The rule is not "never be stateful", it is "keep state in the few places designed to hold it".
Local caches are fine
Keeping a copy of shared data in a server's memory, such as configuration or a popular item, is a performance optimization, not hidden state, as long as losing it only costs a refetch. The danger is state that exists only on that server.
Trade-offs
| Option | Wins when | Loses when |
|---|---|---|
| Sticky sessions | Legacy stateful apps that need to scale out quickly | Even load, deploys and failures: users on a lost server lose their sessions |
| Central session store | Most web apps: any server serves anyone, sessions can be revoked instantly | An extra network hop per request and a critical store to keep available |
| Signed client tokens | High scale and many services: no lookup, works across services | Instant logout or permission changes; tokens stay valid until they expire |
Common mistakes
- Storing uploads on the app server's local disk. The next request lands on a server that does not have the file.
- Relying on sticky sessions and being surprised when a deploy logs everyone out.
- Putting large or sensitive data in client tokens; they are signed, not necessarily encrypted, and travel with every request.
- Calling a service stateless while it keeps in-memory counters for rate limits or jobs that silently diverge across servers.
Check yourself
-
What makes a service stateless?
Show answer
B. It keeps no client data in its own memory between requests Stateless services store state elsewhere, so any instance can handle any request.
-
An app keeps sessions in server memory and you add a second server. What happens?
Show answer
B. Users are randomly logged out when requests hit the other server The other server has no record of the session, so the user appears logged out.
-
What is the main weakness of signed client-side session tokens?
Show answer
B. They are hard to revoke before they expire Servers accept any validly signed, unexpired token, so instant revocation needs extra machinery.
-
Where should user-uploaded images go in a horizontally scaled app?
Show answer
B. Object storage shared by all servers Shared object storage makes the file reachable from every server and from a CDN.
Recap
- State is what a system remembers between requests; every product has some.
- State in one server's memory breaks when traffic moves or the server dies.
- Sessions can be sticky, kept in a shared store, or carried in signed tokens.
- Stateless app servers are interchangeable, which makes scaling and failover easy.
- State moves to databases, caches, object storage and queues built to hold it.
Next up
- Crack the system design round learning path