Skip to content
learn
  1. Learn
  2. High Level System Design
  3. Core Concepts and Back-of-the-Envelope Math

System Design HLD-02.02

Availability, reliability and the nines

What 99.9 percent really allows, why chains of services multiply into lower availability, how redundancy raises it, and MTBF versus MTTR.

Beginner 10 min Last reviewed

The big idea

AvailabilityThe fraction of time, or of requests, for which a system works correctly. Often written as a number of nines, such as 99.9 percent. is the share of time a system does its job. It is usually quoted in "nines": 99.9 percent is three nines, 99.99 percent is four. The numbers look almost identical, but each extra nine cuts allowed downtime by a factor of ten, and usually costs far more than ten times as much to reach.

Two simple rules govern how the availability of parts becomes the availability of the whole. Parts that all must work, in a chain, multiply into something worse than any single part. Spare copies that can stand in for each other multiply their failure chances instead, and become far better than any single copy. Almost every reliability design is an application of those two rules.

Think of a string of old holiday lights wired in series: one bulb fails and the whole string goes dark, so a long string is fragile. Wire two strings in parallel and the room stays lit unless both fail at once.

How it works

Step 1: What the nines allow

Allowed downtime for each availability level Bars on a log scale showing downtime per year: 99 percent allows 3.65 days, 99.5 percent 1.83 days, 99.9 percent 8.76 hours, 99.95 percent 4.38 hours, 99.99 percent 52.6 minutes, 99.999 percent 5.26 minutes. Per 30 days: 7.2 hours, 3.6 hours, 43 minutes, 22 minutes, 4.3 minutes and 26 seconds. downtime per yearper 30 days 99% (two nines) 3.65 days 7.2 hours 99.5% 1.83 days 3.6 hours 99.9% (three nines) 8.76 hours 43 minutes 99.95% 4.38 hours 22 minutes 99.99% (four nines) 52.6 minutes 4.3 minutes 99.999% (five nines) 5.26 minutes 26 seconds Figure 1. Each extra nine divides allowed downtime by ten. At four nines, a single 10 minute incident uses more than two months' budget.

Availability is defined as

or, for request-based services, successful requests divided by all requests. Five minutes of downtime a year is less time than it takes most people to notice an alert, open a laptop and log in, which is why five nines requires fully automatic recovery.

Step 2: In series, availabilities multiply

If a request must pass through several components and each must work, the system is up only when all are up. For independent failures:

Five services at 99.9 percent each give : about 99.5 percent, or roughly 44 hours of downtime a year instead of 9. Every dependency you add to the request path lowers the ceiling.

Step 3: In parallel, failure chances multiply

If you run several copies and any one can serve the request, the system fails only when every copy fails at once. With failure chance per copy:

Two copies at 99 percent each: , four nines from two two-nines machines. That is the whole point of redundancy.

Series availability falls; parallel availability rises Top: three components in a chain, each 99.9 percent, give 99.7 percent overall, drawn with fading glow. Bottom: two copies of a 99 percent component in parallel behind a load balancer give 99.99 percent, drawn with strong glow. In series: all must work In parallel: any one is enough API 99.9% Auth 99.9% Database 99.9% = 99.7% Failover load balancer Copy 1 99% Copy 2 99% = 99.99% Figure 2. Top, a chain is weaker than its weakest link: 0.999 cubed is about 99.7 percent. Bottom, two 99 percent copies with failover reach 99.99 percent, as long as their failures are independent.

Step 4: Fail less often, or recover faster

Two numbers describe failures over time. MTBFMean time between failures: the average time a component runs before it fails. is how long things run between failures; MTTRMean time to recovery: the average time from a failure until service is restored, including detection, diagnosis and the fix. is how long each failure lasts. Availability is

Halving MTTR helps exactly as much as doubling MTBF, and it is often far cheaper: better alerts, automatic failover, one-click rollback and practised runbooks shrink recovery time without needing perfect hardware or bug-free code.

Try it

This interactive needs JavaScript. The diagrams and text above cover the same idea.

Try this:

  1. Four services at 99.9 percent, one copy each. The chain is about 99.6 percent: a day and a half of downtime a year.
  2. Raise the chain to 8 services. Every dependency costs you.
  3. Now set copies to 2. Each service becomes about six nines, and even the 8-service chain beats any single service. That is why every tier in a serious system is redundant.

Going deeper

The independence assumption

The parallel formula assumes copies fail independently. Two servers in the same rack share a power supply and a switch; two copies running the same buggy release fail together on the same input. Real redundancy needs separate failure domains: different racks, availability zones or regions, and staggered deployments. Correlated failures are the reason real systems rarely reach their paper availability. More in Redundancy, failover and multi-AZ .

Availability is not the only reliability

A system that answers every request with wrong data is "available". Reliability also covers durabilityStored data is not lost once it has been acknowledged as written. (acknowledged data is never lost) and correctness. Storage services often quote durability separately, with many more nines than availability, because losing data is worse than being briefly unreachable.

Trade-offs

OptionWins whenLoses when
Add redundancy (parallel copies) Critical tiers: survives individual machine, zone or region failures Cost, and only helps if failures are independent
Shorten the request path Every service: fewer serial dependencies raise the ceiling Teams that need separate services for ownership or scaling
Invest in faster recovery (MTTR) Almost any system: often the cheapest availability gain Failures that lose data, where recovering fast does not undo the damage

Common mistakes

  • Promising more nines than your dependencies have. A service cannot be more available than the chain it requires.
  • Counting redundant copies in the same rack or zone as independent.
  • Treating 99.9 and 99.99 as similar. The second allows one tenth of the downtime and usually costs much more.
  • Measuring availability only from inside the data center. Users experience DNS, the network and the client too.

Check yourself

  1. Roughly how much downtime per year does 99.9 percent availability allow?

    Show answer

    C. About 8.8 hours 0.1 percent of 8,760 hours is about 8.8 hours.

  2. A request passes through three services at 99 percent each. What is the overall availability?

    Show answer

    B. About 97 percent In series, multiply: 0.99^3 is about 0.970.

  3. Two independent copies at 99 percent each, with failover. What is the availability?

    Show answer

    C. 99.99 percent Both must fail: 0.01 x 0.01 = 0.0001, so availability is 99.99 percent.

  4. A service fails about once a month and takes 2 hours to recover. Which change helps most cheaply?

    Show answer

    B. Automating failover so recovery takes 5 minutes Cutting MTTR from 2 hours to 5 minutes reduces downtime 24 times, usually far cheaper than halving failures.

Recap

  • Each extra nine divides allowed downtime by ten: 99.9 percent is about 8.8 hours a year.
  • Components in series multiply availabilities, so every dependency lowers the ceiling.
  • Redundant copies multiply failure chances, so two 99 percent copies give 99.99 percent.
  • Redundancy only works if failures are independent: use separate failure domains.
  • Availability = MTBF / (MTBF + MTTR); faster recovery is often the cheapest win.

Get new lessons by email

Get new posts on backend and AI internals. No spam, unsubscribe anytime.

Your email is only used to send new posts. Privacy

Next up