System Design HLD-02.05
Vertical vs horizontal scaling
Scale up with a bigger machine or scale out with more machines: the limits, costs and requirements of each, and why most systems do both.
The big idea
When a server cannot keep up, there are two ways to get more capacity. Vertical scalingMaking one machine more powerful: more CPU cores, memory, faster disks. Also called scaling up. replaces it with a bigger machine. Horizontal scalingAdding more machines and spreading the work across them. Also called scaling out. adds more machines of the same size and splits the work between them.
Vertical scaling is wonderfully simple: nothing in your code changes. But machines only come so big, the biggest ones cost disproportionately more, and one machine is still one machine when it fails. Horizontal scaling grows almost without limit and survives failures, but it only works if the work can be split, and splitting work brings the hard problems of distributed systems.
Picture a bakery that cannot keep up. It can buy a bigger oven (vertical), which is easy until there are no bigger ovens to buy. Or it can open more bakeries (horizontal), which can grow forever but needs a way to send customers to the right bakery and to share the recipe book.
How it works
Step 1: Scaling up
Moving from 4 cores to 32 needs no code changes, no new failure modes and no coordination between machines. For many systems that is exactly right: a single modern server with dozens of cores and hundreds of gigabytes of memory handles far more traffic than most products ever see. Databases in particular are often scaled vertically first, because splitting them is hard.
The limits are real, though. There is a largest machine you can buy or rent. Upgrading usually means downtime or a failover. And one machine, however large, is a single point of failure.
Step 2: Scaling out
Horizontal scaling puts identical servers behind a load balancerA component that receives incoming requests and spreads them across a pool of servers, skipping unhealthy ones.. Need more capacity? Add servers. One dies? The load balancer stops sending it traffic. This is how every large web service runs its application tier. It requires three things:
- Statelessness: any server must be able to handle any request, so user sessions and data live in shared stores, not in one server's memory. That is the next lesson, Stateless vs stateful services .
- Load balancing: something must spread traffic and detect failed servers. See What a load balancer does .
- Data partitioning: when the data itself outgrows one machine, it must be split across many, which is much harder than adding app servers. See Partitioning and sharding .
Step 3: The cost curves
Going deeper
Diagonal scaling
Real systems mix both. Teams scale a database vertically as far as is comfortable, while scaling stateless app servers horizontally from day one. Some also scale out with fewer, larger machines rather than many tiny ones, which reduces coordination and per-machine overhead. Cloud autoscaling usually adds and removes machines horizontally, because that can happen without downtime.
Horizontal is not free
Once work is spread over machines, you meet partial failures, network delays between nodes and data that must agree across copies. These are the subject of the whole distributed systems module starting with Why distributed systems are hard . Do not pay that price before you need to.
Trade-offs
| Option | Wins when | Loses when |
|---|---|---|
| Vertical scaling | Early stages, databases, workloads that are hard to split: simple and fast to do | Beyond the largest machine, high availability, and cost at the top end |
| Horizontal scaling | Stateless app tiers, very large scale, surviving machine failures | Stateful or tightly coupled workloads: needs statelessness, balancing and partitioning |
| Both (diagonal) | Most real systems: scale app servers out, scale the database up until you must shard | Nothing in particular, but you must know when each stops working |
Common mistakes
- Sharding a database early when a bigger machine would carry you for years.
- Scaling out an app tier that keeps sessions in memory, so users are logged out at random.
- Forgetting that a single big machine is a single point of failure, however reliable the hardware.
- Assuming twice the machines gives twice the throughput. Shared bottlenecks, such as one database, cap the gain.
Check yourself
-
What is vertical scaling?
Show answer
B. Moving to a more powerful machine Vertical scaling, or scaling up, makes one machine bigger.
-
Which requirement does horizontal scaling of app servers depend on most?
Show answer
B. Servers being stateless, so any server can handle any request If servers hold user state in memory, requests cannot freely move between them.
-
Why are databases often scaled vertically first?
Show answer
B. Splitting data across machines is complex, while a bigger machine changes nothing in the code Partitioning data brings hard problems; a bigger machine postpones them cheaply.
-
You doubled app servers from 10 to 20, but throughput barely changed. What is the likely cause?
Show answer
B. A shared bottleneck such as the database Adding app servers only helps if they were the bottleneck; a saturated shared database caps everything.
Recap
- Vertical scaling buys a bigger machine: simple, but capped and a single point of failure.
- Horizontal scaling adds machines behind a load balancer: nearly unlimited and fault tolerant.
- Scaling out needs stateless services, load balancing and, for data, partitioning.
- Top-end hardware costs more per unit of capacity; many machines add fixed overhead.
- Most systems scale app servers out and databases up, until they must shard.
Next up
- I know nothing yet learning path
- Crack the system design round learning path