Skip to content
learn
  1. Learn
  2. High Level System Design
  3. Reliability and Resilience

System Design HLD-12.06

Multi-region architecture

Why run in several regions (latency, disaster recovery, data residency), how to replicate data across them, pin users to home regions, and what it costs.

Advanced 15 min Last reviewed

The big idea

Multiple availability zones protect you from losing a building. They do not protect you from losing a whole region: a regional network or control-plane failure, a natural disaster, or a bad change that hits every zone in the region. And they do nothing for a user in Sydney whose every request crosses the Pacific to Virginia and back. Running in several regionsA cloud provider's geographic area, such as eu-west or ap-southeast, containing several availability zones; regions are far apart and largely independent. addresses three needs: latency for users around the world, disaster recovery from a regional outage, and data residency laws that require some data to stay in a country or bloc.

Stateless services are easy to copy to another region. The hard part is always data. Regions are thousands of kilometers apart, so synchronous replication between them adds tens to hundreds of milliseconds to every write, and asynchronous replication means a region failure can lose recent writes or let two regions accept conflicting ones. Every multi-region design is a choice about where each piece of data lives and who may write it.

Think of a company with offices on three continents. Each office serves its local customers quickly. But the customer records cannot be in three places at once and be identical instantly; either one office owns each customer's file and others ask it, or every office keeps a copy and they reconcile disagreements.

How it works

Step 1: Distance is the constraint

regions.pyruns with python3 (standard library)
# Light in fiber covers about 200 km per millisecond, so distance sets a hard floor on latency.
KM = {("virginia", "ireland"): 5_500, ("virginia", "singapore"): 15_500,
      ("ireland", "singapore"): 11_000}
def rtt(a, b):
    d = KM.get((a, b)) or KM.get((b, a))
    return 2 * d / 200                                # best case; real routes add 30 to 60%

regions = ["virginia", "ireland", "singapore"]
for i, a in enumerate(regions):
    for b in regions[i + 1:]:
        print(f"{a:>9} <-> {b:<9} round trip >= {rtt(a, b):5.0f} ms")

# A write in Virginia that must be confirmed synchronously elsewhere before replying:
print("sync to all regions     : +", max(rtt("virginia", r) for r in regions[1:]), "ms per write")
print("majority of 3 (1 other) : +", min(rtt("virginia", r) for r in regions[1:]), "ms per write")
print("async replication       : + 0 ms per write, but recent writes are lost if Virginia dies")

Output

 virginia <-> ireland   round trip >=    55 ms
 virginia <-> singapore round trip >=   155 ms
  ireland <-> singapore round trip >=   110 ms
sync to all regions     : + 155.0 ms per write
majority of 3 (1 other) : + 55.0 ms per write
async replication       : + 0 ms per write, but recent writes are lost if Virginia dies

Light in optical fiber travels about 200 km per millisecond, so physics alone puts a round trip from Virginia to Singapore above 150 ms; real routes add more. Waiting for every region to confirm each write costs that on every write. Waiting for a majority of three regions costs the round trip to the nearest other region. Asynchronous replication costs nothing per write but risks losing the last moments of writes if a region dies, the PACELC trade-off in action ( PACELC and consistency models ).

Step 2: Getting users to the right region

Three regions with users routed to the nearest and data replicated between them Three regions laid out like a world map: us-east, eu-west and ap-southeast. Users in each continent are routed by DNS or anycast to their nearest region. Each region holds its home users' data as the writable copy. Glowing links between regions carry asynchronous replication for disaster recovery and read replicas. async replication us-east home: Americas users eu-west home: EU users (residency) ap-southeast home: APAC users Americas Europe Asia-Pacific Figure 1. Users are routed to their nearest region. Each user's data is written in their home region and replicated asynchronously to the others, so reads are local and a region failure can be survived by promoting a copy.

Traffic is steered with latency-based or geographic DNS, which answers each user with the address of a nearby region, or with anycast, where the same IP address is announced from many locations and the internet routes to the closest ( Global traffic: GeoDNS, anycast and GSLB ). Both can shift traffic away from a failed region, DNS more slowly because of caching.

Step 3: Patterns for data

  • Active-passive (disaster recovery): one primary region takes all writes and replicates asynchronously to a standby region. Simple, no conflicts. On a regional outage, promote the standby, accepting a small amount of data loss ( Disaster recovery: RPO and RTO ). Users far from the primary get no latency benefit for writes.
  • Read local, write home: every region has read replicas for fast reads, but writes go to one primary region. Good for read-heavy products; writes from far away are slow, and readers may see replication lag.
  • Region pinning (home region per user): each user or tenant is assigned a home region where their data is written, chosen by location or contract. Their reads and writes are local and fast, and data residency comes almost for free. Cross-user features, such as messaging someone in another region, need cross-region calls, and moving a user's home is a migration.
  • Multi-leader, active-active: every region accepts writes to the same data and replicates to the others asynchronously. Lowest latency everywhere, but concurrent writes to the same record in two regions conflict and must be resolved by last-writer-wins, merging, or CRDTs ( Multi-leader and leaderless replication ). Suits data that is naturally mergeable or rarely edited from two places at once.
  • Globally consistent databases: systems such as Spanner and CockroachDB run consensus across regions to give strong consistency, at the cost of cross-region latency on writes, which they reduce by placing each piece of data's leader near where it is written.

Most real systems mix these: user profiles pinned to a home region, product catalog replicated everywhere read-only, payments in one consistent primary.

Going deeper

Failover is harder than it looks

Regional failover moves a large share of all traffic onto other regions at once, so they need spare capacity ready: running each of three regions at no more than about two-thirds of its capacity ( Redundancy, failover and multi-AZ ). Failing over too eagerly on a partial problem can cause more damage than waiting, so many teams make it a deliberate, rehearsed human decision with automated tooling, and fail back carefully once the region recovers and data has been reconciled.

The cost

Multi-region roughly multiplies infrastructure cost by the number of regions, adds cross-region data transfer charges, and multiplies operational work: deploys, monitoring, on-call and testing in every region, plus the logic to handle cross-region data. Many products are better served by a single region with multiple zones and a well-tested disaster recovery plan.

Trade-offs

OptionWins whenLoses when
Single region, multi-AZ Most products; simple and much cheaper Regional outages; latency for distant users
Active-passive DR region Surviving a regional disaster with modest complexity Some data loss on failover; no latency benefit for writes
Region pinning per user Global apps with data residency needs and mostly local interactions Cross-region features and moving users between regions
Multi-leader active-active Low-latency writes everywhere for mergeable data Write conflicts and complex reasoning
Globally consistent database Data that must be correct everywhere, such as balances Cross-region latency on writes; cost

Common mistakes

  • Going multi-region for availability while keeping a single-region dependency (auth, a database, DNS config) on the critical path.
  • Assuming synchronous replication across continents will be fast enough.
  • Running multi-leader writes on data that cannot tolerate conflicts, such as balances or inventory.
  • Not keeping spare capacity in the surviving regions for a failover.
  • Never rehearsing regional failover, so the first attempt happens during a real disaster.

In the wild

Large consumer services such as video streaming, social networks and global SaaS products run in many regions and route users to the nearest one. Cloud databases offer the main patterns as products: Amazon DynamoDB global tables provide multi-active replication across regions, while Google Spanner and CockroachDB provide strongly consistent multi-region databases. Data residency requirements such as the EU's GDPR are a frequent reason SaaS companies pin customer data to a home region.

Check yourself

  1. Why can't synchronous replication between Virginia and Singapore be made fast with better hardware?

    Show answer

    B. The speed of light in fiber sets a round-trip floor above 150 ms over that distance Physics bounds the latency; only less distance or fewer synchronous round trips help.

  2. A SaaS company must keep EU customers' data in the EU. Which pattern fits most naturally?

    Show answer

    B. Region pinning: each customer has a home region where their data lives Pinning tenants to a home region puts their data where the law requires and keeps their requests local.

  3. Two regions accept writes to the same user profile concurrently with asynchronous multi-leader replication. What must the design handle?

    Show answer

    B. Write conflicts, resolved by last-writer-wins, merging or CRDTs Concurrent updates in different regions diverge and must be reconciled.

  4. With asynchronous replication to a standby region, what happens to the most recent writes if the primary region is lost?

    Show answer

    B. Writes not yet replicated are lost, bounded by replication lag Async replication trades a small data-loss window (RPO) for no added write latency.

Recap

  • Multi-region serves global latency, regional disaster recovery and data residency.
  • Distance makes synchronous cross-region writes slow; asynchronous replication risks losing recent writes.
  • DNS or anycast routes users to nearby regions and away from failed ones.
  • Data patterns: active-passive DR, read local write home, region pinning, multi-leader, global consensus databases.
  • It multiplies cost and operations; keep spare capacity and rehearse failover.

Get new lessons by email

Get new posts on backend and AI internals. No spam, unsubscribe anytime.

Your email is only used to send new posts. Privacy