Skip to content
learn
  1. Learn
  2. High Level System Design
  3. Core Concepts and Back-of-the-Envelope Math

System Design HLD-02.06

Stateless vs stateful services

What state is, why servers that remember users break when traffic moves, and where to put sessions so any server can handle any request.

Beginner 10 min Last reviewed

The big idea

StateData a system remembers between requests, such as who is logged in, what is in a cart or how far a long upload has got. is anything a system remembers from one request to the next: that you are logged in, what is in your cart, which step of checkout you reached. Every useful product has state. The design question is where it lives.

If a server keeps state in its own memory, the user must keep coming back to that exact server. The moment that server restarts, is replaced or the load balancer sends the user elsewhere, the state is gone: the user is logged out or the cart is empty. A statelessA service that keeps no client data in its own memory between requests, so any instance can handle any request. service instead keeps nothing between requests and stores state in a shared place every server can reach. Then servers become interchangeable, which is exactly what horizontal scaling and failover need.

Picture a coffee shop with several baristas. If your order is only in one barista's head, you must wait for that barista. If it is written on a ticket on the shared rail, any barista can make it, and nothing is lost when one goes on break.

How it works

Step 1: The stateful failure

A stateful server loses the session when traffic moves; a shared session store fixes it Top: the user logs in on server A, which keeps the session in memory. The next request goes to server B, which has no session, so the user appears logged out, highlighted red. Bottom: both servers read sessions from a shared store, so the request to server B succeeds, highlighted green. Stateful: session in server memory Stateless: session in a shared store request 1 request 2 User Load balancer Server A session: alice Server B who are you? logged out User Load balancer Server A Server B Session store Redis or DB Figure 1. Top, the session lives in server A's memory, so a request routed to B fails (red). Bottom, sessions live in a shared store and any server can answer (green).

Step 2: Three places to keep sessions

Sticky sessions. The load balancer pins each user to one server, using a cookie or the client's IP. It is a quick fix for an existing stateful app, but load spreads unevenly, and when the server dies its users still lose their sessions.

A central session store. Servers keep nothing locally and read the session from a fast shared store, usually an in-memory database like Redis, on every request. Any server can serve anyone. The store becomes a critical dependency, so it needs replication, but it is a single, well-understood piece.

Client-side tokens. The server puts the session data, such as user ID and expiry, into a signed tokenA piece of data signed by the server with a secret key, so the server can later verify it was not altered. JWTs are a common format. that the client sends back with each request. Servers verify the signature and need no lookup at all. The catch is revocation: a token stays valid until it expires unless you add a block list, so tokens are usually short-lived. See OAuth 2.0, OpenID Connect and JWT .

Step 3: State still has to live somewhere

Statelessness does not remove state, it moves it to systems built for it:

  • Databases for durable data: accounts, orders, posts.
  • Caches and session stores for fast, short-lived data.
  • Object storage for files and media.
  • Queues for work in progress.

The application tier becomes a pool of interchangeable workers that you can add, remove, restart and deploy freely, while the hard problems of keeping state correct are concentrated in a few specialized systems.

Try it

Where should each piece of state live in a scaled-out web app?

Show answers
  • A user's order history: Database. Durable business data that must survive any failure.
  • The login session for the current visit: Cache or session store. Read on every request, short-lived, and must be reachable from every server.
  • Profile photos: Object storage. Large binary files belong in object storage, usually served through a CDN.
  • A compiled template reused by every request: Fine in server memory. Identical on every server and rebuilt on restart, so it is not user state at all.
  • The product page shown 10,000 times a minute: Cache or session store. A shared cache saves the database from repeating the same read.
  • Items in a shopping cart: Database. Users expect carts to survive across devices and days, so they need durable storage, often fronted by a cache.

Going deeper

Some services are stateful on purpose

Databases, caches, message brokers and real-time systems like chat servers holding open WebSocket connections are inherently stateful. They are scaled with techniques of their own: replication, partitioning and consistent routing of each user or key to the right node. You will meet these from Replication: leader-follower onward. The rule is not "never be stateful", it is "keep state in the few places designed to hold it".

Local caches are fine

Keeping a copy of shared data in a server's memory, such as configuration or a popular item, is a performance optimization, not hidden state, as long as losing it only costs a refetch. The danger is state that exists only on that server.

Trade-offs

OptionWins whenLoses when
Sticky sessions Legacy stateful apps that need to scale out quickly Even load, deploys and failures: users on a lost server lose their sessions
Central session store Most web apps: any server serves anyone, sessions can be revoked instantly An extra network hop per request and a critical store to keep available
Signed client tokens High scale and many services: no lookup, works across services Instant logout or permission changes; tokens stay valid until they expire

Common mistakes

  • Storing uploads on the app server's local disk. The next request lands on a server that does not have the file.
  • Relying on sticky sessions and being surprised when a deploy logs everyone out.
  • Putting large or sensitive data in client tokens; they are signed, not necessarily encrypted, and travel with every request.
  • Calling a service stateless while it keeps in-memory counters for rate limits or jobs that silently diverge across servers.

Check yourself

  1. What makes a service stateless?

    Show answer

    B. It keeps no client data in its own memory between requests Stateless services store state elsewhere, so any instance can handle any request.

  2. An app keeps sessions in server memory and you add a second server. What happens?

    Show answer

    B. Users are randomly logged out when requests hit the other server The other server has no record of the session, so the user appears logged out.

  3. What is the main weakness of signed client-side session tokens?

    Show answer

    B. They are hard to revoke before they expire Servers accept any validly signed, unexpired token, so instant revocation needs extra machinery.

  4. Where should user-uploaded images go in a horizontally scaled app?

    Show answer

    B. Object storage shared by all servers Shared object storage makes the file reachable from every server and from a CDN.

Recap

  • State is what a system remembers between requests; every product has some.
  • State in one server's memory breaks when traffic moves or the server dies.
  • Sessions can be sticky, kept in a shared store, or carried in signed tokens.
  • Stateless app servers are interchangeable, which makes scaling and failover easy.
  • State moves to databases, caches, object storage and queues built to hold it.

Get new lessons by email

Get new posts on backend and AI internals. No spam, unsubscribe anytime.

Your email is only used to send new posts. Privacy

Next up