System Design HLD-04.05
Global traffic: GeoDNS, anycast and GSLB
Sending each user to a nearby healthy region with GeoDNS, anycast and global load balancers, and failing over when a whole region goes down.
The big idea
A load balancer spreads traffic within one data center. But if your servers are in Virginia and your user is in Singapore, every round trip crosses the Pacific: around 200 ms before your code even runs ( Latency numbers every engineer should know ). And if the whole Virginia region has an outage, a load balancer inside it cannot help. Serious global services therefore run in several regions and add a layer above load balancing: global traffic managementSteering each user to an appropriate region or point of presence, based on location, latency, health and capacity., which decides which region each user should reach in the first place.
Three tools do this. GeoDNS gives different DNS answers to users in different places. Anycast announces the same IP address from many locations, so the internet itself routes each user to the nearest one. Global server load balancing (GSLB) combines these with health checks and live latency measurements, and moves traffic away from a region that fails.
Think of a pizza chain with one phone number. Whoever calls is connected to the nearest branch. If a branch's oven breaks, calls from that neighborhood are quietly routed to the next nearest one.
How it works
Step 1: GeoDNS
The authoritative DNS server looks at where a query comes from, usually the IP address of the user's recursive resolver, and answers with the address of the closest healthy region: api.example.com resolves to a Frankfurt IP for a user in Berlin and to a Tokyo IP for a user in Osaka. Because it is just DNS, it works with any protocol. Its weaknesses come from DNS itself: the resolver's location is only a guess at the user's, and cached answers keep users on a region for up to the TTL after something changes.
Step 2: Anycast
With anycastAdvertising the same IP address from many locations via BGP, so internet routers deliver each user's packets to the topologically nearest location., the same IP address is advertised by routers in many cities using BGP, the internet's routing protocol. Each user's packets go to whichever location is nearest in network terms. Nothing on the client changes and nothing is cached: if a location stops advertising the address, routes converge on the next nearest location, often within seconds to a minute. CDNs and public DNS resolvers rely on anycast, and it naturally spreads attack traffic across all locations instead of concentrating it.
Step 3: GSLB and latency-based routing
A global load balancer runs health checks against every region and keeps a picture of their latency and capacity, often from real user measurements. It answers each user, by DNS or by anycast plus a global proxy layer, with the best region by some policy: lowest latency, nearest healthy, a weighted split for gradual migrations, or "all traffic to region A, B only as backup" for active-passive setups. Cloud providers offer this as managed global load balancing.
Step 4: Failing over a region
A full regional failover is more than routing. The surviving regions must have spare capacity for the extra users (or autoscale fast enough), the data those users need must already be in the other region ( Multi-region architecture ), and the switch should be practised regularly, because a failover path that is never tested usually fails when needed.
Going deeper
How fast is failover?
With anycast, withdrawal of a route can propagate in seconds. With DNS-based steering, users follow at the pace of TTLs and resolver caches, so services that rely on it keep TTLs short (30 to 60 seconds) and still see a long tail of clients on the old answer. Many large providers combine them: anycast to a nearby edge, then a private backbone and their own routing to the right region.
Anycast and TCP
Because routes can change mid-connection, anycast was long considered risky for long-lived TCP. In practice, routing is stable enough for typical web requests, and operators avoid flapping routes. For long-lived sessions, an anycast front door that proxies to a stable unicast backend is common.
Trade-offs
| Option | Wins when | Loses when |
|---|---|---|
| GeoDNS | Simple region steering for any protocol with standard DNS | Precision (resolver location) and speed of change (TTL caching) |
| Anycast | Fast, cache-free nearest routing; absorbing attacks; CDNs and DNS | Fine control over which location serves whom, and stateful long connections |
| Managed GSLB | Health, latency and capacity aware routing with policies | Cost and dependence on one provider's global layer |
Common mistakes
- Running in several regions without the spare capacity to absorb a failed region's users.
- Relying on DNS failover with hour-long TTLs.
- Steering users to a region that does not have their data, turning a routing fix into errors.
- Never testing regional failover until a real outage.
Check yourself
-
How does anycast send users to the nearest location?
Show answer
B. The same IP is advertised from many places, and internet routing delivers packets to the nearest BGP routes each user's packets to the topologically nearest location announcing the address.
-
Why is GeoDNS failover not instant?
Show answer
B. Resolvers and clients cache answers for the TTL Users keep the cached old answer until it expires, and some caches hold it longer.
-
Frankfurt fails and its users go to Virginia. What must already be true for this to work well?
Show answer
A. Virginia has spare capacity and the users' data is available there Routing is only half: the target region needs capacity and the data.
-
What does a GSLB add beyond plain GeoDNS?
Show answer
B. Health, latency and capacity awareness with routing policies It combines location with live health and performance to pick the best region.
Recap
- Global traffic management picks a region for each user before load balancing starts.
- GeoDNS answers by location; it is simple but limited by resolver location and TTLs.
- Anycast advertises one IP from many places; routing picks the nearest, with fast failover.
- GSLB combines health, latency and capacity with routing policies.
- Regional failover also needs spare capacity and data in the surviving regions.
Next up
- Crack the system design round learning path