System Design HLD-16.04
Multi-tenancy and isolation
Serving many customers on shared infrastructure: pooled, schema-per-tenant and dedicated models, row-level security, noisy neighbours and per-tenant limits.
The big idea
A software-as-a-service product serves 5,000 companies. Running a separate copy of everything for each would be enormously expensive and impossible to operate, so customers share servers and databases. That raises two hard requirements. Data isolation: one company must never see another's data, even through a bug. Performance isolation: one company's huge import job must not slow everyone else down. A system serving many customers this way is multi-tenantServing many customers (tenants) from shared infrastructure while keeping each tenant's data and performance isolated from the others., and each customer is a tenant.
There is a spectrum of designs, from sharing everything with a tenant id on every row, to separate schemas or databases per tenant, to fully dedicated stacks for the largest or most regulated customers. Most mature products mix them.
Think of places to live. A hotel room shares the building, staff and plumbing with everyone, and is cheap. An apartment has its own locked unit but shares the structure. A detached house is fully yours and costs the most. Different customers want, and pay for, different levels of separation.
How it works
Step 1: Three tenancy models
- Pooled: shared app and shared tables, with a
tenant_idcolumn on every row. Cheapest and simplest to operate at scale; isolation depends entirely on every query filtering correctly. - Bridge: a mix of the two, for example a shared app tier with a separate schema or database per tenant. Stronger data separation, easier per-tenant backup, restore and deletion, but thousands of schemas make migrations slow and connection pools large.
- Silo: dedicated stack per tenant, often in its own account or cell ( Cell-based architecture and shuffle sharding ). Maximum isolation and customization, highest cost and operational burden; typically reserved for the largest or most regulated customers.
Step 2: Data isolation
In the pooled model, one forgotten WHERE tenant_id = ? leaks data across customers. Make the filter impossible to forget:
import sqlite3
db = sqlite3.connect(":memory:")
db.executescript("""
CREATE TABLE invoices (tenant_id TEXT, id INTEGER, amount INTEGER);
INSERT INTO invoices VALUES ('acme', 1, 100), ('acme', 2, 250), ('globex', 1, 9000);
""")
class TenantDB:
"""Every query is scoped to one tenant; callers cannot forget the filter."""
def __init__(self, conn, tenant):
self.conn, self.tenant = conn, tenant
def invoices(self, where="1=1", args=()):
sql = f"SELECT id, amount FROM invoices WHERE tenant_id = ? AND ({where})"
return self.conn.execute(sql, (self.tenant, *args)).fetchall()
print("acme sees :", TenantDB(db, "acme").invoices())
print("globex sees:", TenantDB(db, "globex").invoices())
# A bug or attack asks for invoice id 1 from acme's session: still only acme's row.
print("acme asks for id 1:", TenantDB(db, "acme").invoices("id = ?", (1,)))
# Performance isolation: a per-tenant quota so one tenant cannot use all the capacity.
QUOTA = {"acme": 100, "globex": 100} # requests per second per tenant
offered = {"acme": 40, "globex": 900} # globex runs a runaway batch job
served = {t: min(r, QUOTA[t]) for t, r in offered.items()}
print("served per tenant:", served, "| acme unaffected by globex's spike")Output
acme sees : [(1, 100), (2, 250)]
globex sees: [(1, 9000)]
acme asks for id 1: [(1, 100)]
served per tenant: {'acme': 40, 'globex': 100} | acme unaffected by globex's spike - Derive the tenant from the authenticated identity, never from a request parameter the client can change ( Authentication vs authorization ).
- Route all data access through a tenant-scoped layer, as above, so the filter is applied automatically.
- Add a database-enforced backstop: PostgreSQL row-level security policies, for example, make the database itself hide other tenants' rows based on a session setting.
- Include the tenant in cache keys, search indexes, file paths and encryption keys. Leaks often happen in the cache, not the database.
Step 3: Performance isolation
A noisy neighbourA tenant whose heavy usage consumes shared resources and degrades performance for other tenants on the same infrastructure. runs a huge report or import and saturates the shared database. Defenses:
- Per-tenant rate limits and quotas on requests, concurrency, storage and background jobs ( Rate limiting algorithms ).
- Fair scheduling of shared queues and workers, so one tenant's backlog does not starve others.
- Placement: move heavy tenants to their own shard, cell or silo; use shuffle sharding so a bad tenant hurts very few others ( Cell-based architecture and shuffle sharding ).
- Per-tenant metrics, so you can see who is consuming what and bill or limit accordingly.
Going deeper
Tiering
Many products offer tiers that map onto these models: small customers pooled, mid-size customers in pooled infrastructure with stronger quotas, and enterprise customers in dedicated cells or silos with their own encryption keys and data residency ( Multi-region architecture ). Design the tenant id, routing and migration tooling so a tenant can move between tiers.
Trade-offs
| Option | Wins when | Loses when |
|---|---|---|
| Pooled (shared tables) | Many small tenants; lowest cost and simplest operations | Isolation depends on every query; noisy neighbours |
| Bridge (schema or database per tenant) | Stronger separation, per-tenant restore and deletion | Migrations across thousands of schemas; connection overhead |
| Silo (dedicated stack) | Large or regulated tenants needing strict isolation | Highest cost; many stacks to deploy and monitor |
| Mixed tiers | Matching isolation to what each customer pays for | Tooling to support and migrate between models |
Common mistakes
- Taking the tenant id from a request parameter instead of the authenticated identity.
- Forgetting the tenant in cache keys, search indexes or object storage paths.
- No per-tenant limits, so one customer's batch job slows everyone.
- Giving every tenant its own database from day one and drowning in migrations.
- No per-tenant metrics, so you cannot tell who is causing load.
In the wild
AWS
SaaS Tenant Isolation Strategies (opens in a new tab)AWS's SaaS Factory whitepaper, by Tod Golding, names the same three models this lesson uses. In the silo model each tenant gets dedicated resources; in the pool model tenants share them side by side; and because real systems are rarely all one or the other, the bridge model mixes the two, for example a pooled web tier in front of per-tenant storage. It also lists a cost of pooling that is easy to miss: in a pool, attributing resource consumption to a specific tenant becomes much harder, so providers must instrument their systems to track per-tenant usage. AWS now marks the paper as kept for historical reference, but its vocabulary is still the common one.
Check yourself
-
Where should the tenant id used in queries come from?
Show answer
B. The authenticated identity of the caller Anything the client controls can be changed to another tenant's id.
-
A pooled system's cache key is the product id only. What can go wrong?
Show answer
B. One tenant can be served another tenant's cached data Tenant ids belong in every key of every shared store, including caches.
-
One tenant's nightly import slows everyone. Which fix addresses it directly?
Show answer
B. Per-tenant quotas and fair scheduling, or moving that tenant to its own shard or cell Performance isolation needs limits and placement per tenant.
-
Which model suits a regulated bank customer requiring dedicated infrastructure and its own keys?
Show answer
B. Silo A dedicated stack gives the strongest isolation, at higher cost.
Recap
- Multi-tenancy serves many customers on shared infrastructure; isolation covers data and performance.
- Pooled, bridge and silo models trade cost and simplicity against isolation; mature products mix them.
- Derive the tenant from identity and scope all data access automatically, with database-level backstops.
- Include the tenant in caches, indexes, paths and keys.
- Per-tenant limits, fair scheduling, placement and metrics contain noisy neighbours.