Skip to content
learn
  1. Learn
  2. High Level System Design
  3. Security in System Design

System Design HLD-16.04

Multi-tenancy and isolation

Serving many customers on shared infrastructure: pooled, schema-per-tenant and dedicated models, row-level security, noisy neighbours and per-tenant limits.

Advanced 10 min Last reviewed

The big idea

A software-as-a-service product serves 5,000 companies. Running a separate copy of everything for each would be enormously expensive and impossible to operate, so customers share servers and databases. That raises two hard requirements. Data isolation: one company must never see another's data, even through a bug. Performance isolation: one company's huge import job must not slow everyone else down. A system serving many customers this way is multi-tenantServing many customers (tenants) from shared infrastructure while keeping each tenant's data and performance isolated from the others., and each customer is a tenant.

There is a spectrum of designs, from sharing everything with a tenant id on every row, to separate schemas or databases per tenant, to fully dedicated stacks for the largest or most regulated customers. Most mature products mix them.

Think of places to live. A hotel room shares the building, staff and plumbing with everyone, and is cheap. An apartment has its own locked unit but shares the structure. A detached house is fully yours and costs the most. Different customers want, and pay for, different levels of separation.

How it works

Step 1: Three tenancy models

Three tenancy models: hotel room, apartment and house Three panels. Pooled, like a hotel: one app and one database shared by tenants A, B and C, with a tenant id on every row. Bridge, like an apartment building: one shared app, but a separate schema or database per tenant. Silo, like a house: tenant D has its own dedicated app and database, glowing. Pooled (hotel room) Bridge (apartment) Silo (house) Shared app tenants A, B, C One database tenant_id on every row Shared app tenants A, B, C schema A B schema C Dedicated app tenant D only Dedicated DB tenant D only Figure 1. Pooled shares everything and relies on a tenant id in every row; bridge shares compute but gives each tenant its own schema or database; silo gives a tenant a dedicated stack. Isolation and cost both rise from left to right.
  • Pooled: shared app and shared tables, with a tenant_id column on every row. Cheapest and simplest to operate at scale; isolation depends entirely on every query filtering correctly.
  • Bridge: a mix of the two, for example a shared app tier with a separate schema or database per tenant. Stronger data separation, easier per-tenant backup, restore and deletion, but thousands of schemas make migrations slow and connection pools large.
  • Silo: dedicated stack per tenant, often in its own account or cell ( Cell-based architecture and shuffle sharding ). Maximum isolation and customization, highest cost and operational burden; typically reserved for the largest or most regulated customers.

Step 2: Data isolation

In the pooled model, one forgotten WHERE tenant_id = ? leaks data across customers. Make the filter impossible to forget:

tenancy.pyruns with python3 (standard library, sqlite3)
import sqlite3

db = sqlite3.connect(":memory:")
db.executescript("""
  CREATE TABLE invoices (tenant_id TEXT, id INTEGER, amount INTEGER);
  INSERT INTO invoices VALUES ('acme', 1, 100), ('acme', 2, 250), ('globex', 1, 9000);
""")

class TenantDB:
    """Every query is scoped to one tenant; callers cannot forget the filter."""
    def __init__(self, conn, tenant):
        self.conn, self.tenant = conn, tenant
    def invoices(self, where="1=1", args=()):
        sql = f"SELECT id, amount FROM invoices WHERE tenant_id = ? AND ({where})"
        return self.conn.execute(sql, (self.tenant, *args)).fetchall()

print("acme sees  :", TenantDB(db, "acme").invoices())
print("globex sees:", TenantDB(db, "globex").invoices())
# A bug or attack asks for invoice id 1 from acme's session: still only acme's row.
print("acme asks for id 1:", TenantDB(db, "acme").invoices("id = ?", (1,)))

# Performance isolation: a per-tenant quota so one tenant cannot use all the capacity.
QUOTA = {"acme": 100, "globex": 100}          # requests per second per tenant
offered = {"acme": 40, "globex": 900}          # globex runs a runaway batch job
served = {t: min(r, QUOTA[t]) for t, r in offered.items()}
print("served per tenant:", served, "| acme unaffected by globex's spike")

Output

acme sees  : [(1, 100), (2, 250)]
globex sees: [(1, 9000)]
acme asks for id 1: [(1, 100)]
served per tenant: {'acme': 40, 'globex': 100} | acme unaffected by globex's spike
  • Derive the tenant from the authenticated identity, never from a request parameter the client can change ( Authentication vs authorization ).
  • Route all data access through a tenant-scoped layer, as above, so the filter is applied automatically.
  • Add a database-enforced backstop: PostgreSQL row-level security policies, for example, make the database itself hide other tenants' rows based on a session setting.
  • Include the tenant in cache keys, search indexes, file paths and encryption keys. Leaks often happen in the cache, not the database.

Step 3: Performance isolation

A noisy neighbourA tenant whose heavy usage consumes shared resources and degrades performance for other tenants on the same infrastructure. runs a huge report or import and saturates the shared database. Defenses:

  • Per-tenant rate limits and quotas on requests, concurrency, storage and background jobs ( Rate limiting algorithms ).
  • Fair scheduling of shared queues and workers, so one tenant's backlog does not starve others.
  • Placement: move heavy tenants to their own shard, cell or silo; use shuffle sharding so a bad tenant hurts very few others ( Cell-based architecture and shuffle sharding ).
  • Per-tenant metrics, so you can see who is consuming what and bill or limit accordingly.

Going deeper

Tiering

Many products offer tiers that map onto these models: small customers pooled, mid-size customers in pooled infrastructure with stronger quotas, and enterprise customers in dedicated cells or silos with their own encryption keys and data residency ( Multi-region architecture ). Design the tenant id, routing and migration tooling so a tenant can move between tiers.

Trade-offs

OptionWins whenLoses when
Pooled (shared tables) Many small tenants; lowest cost and simplest operations Isolation depends on every query; noisy neighbours
Bridge (schema or database per tenant) Stronger separation, per-tenant restore and deletion Migrations across thousands of schemas; connection overhead
Silo (dedicated stack) Large or regulated tenants needing strict isolation Highest cost; many stacks to deploy and monitor
Mixed tiers Matching isolation to what each customer pays for Tooling to support and migrate between models

Common mistakes

  • Taking the tenant id from a request parameter instead of the authenticated identity.
  • Forgetting the tenant in cache keys, search indexes or object storage paths.
  • No per-tenant limits, so one customer's batch job slows everyone.
  • Giving every tenant its own database from day one and drowning in migrations.
  • No per-tenant metrics, so you cannot tell who is causing load.

In the wild

AWS

SaaS Tenant Isolation Strategies (opens in a new tab)

AWS's SaaS Factory whitepaper, by Tod Golding, names the same three models this lesson uses. In the silo model each tenant gets dedicated resources; in the pool model tenants share them side by side; and because real systems are rarely all one or the other, the bridge model mixes the two, for example a pooled web tier in front of per-tenant storage. It also lists a cost of pooling that is easy to miss: in a pool, attributing resource consumption to a specific tenant becomes much harder, so providers must instrument their systems to track per-tenant usage. AWS now marks the paper as kept for historical reference, but its vocabulary is still the common one.

Check yourself

  1. Where should the tenant id used in queries come from?

    Show answer

    B. The authenticated identity of the caller Anything the client controls can be changed to another tenant's id.

  2. A pooled system's cache key is the product id only. What can go wrong?

    Show answer

    B. One tenant can be served another tenant's cached data Tenant ids belong in every key of every shared store, including caches.

  3. One tenant's nightly import slows everyone. Which fix addresses it directly?

    Show answer

    B. Per-tenant quotas and fair scheduling, or moving that tenant to its own shard or cell Performance isolation needs limits and placement per tenant.

  4. Which model suits a regulated bank customer requiring dedicated infrastructure and its own keys?

    Show answer

    B. Silo A dedicated stack gives the strongest isolation, at higher cost.

Recap

  • Multi-tenancy serves many customers on shared infrastructure; isolation covers data and performance.
  • Pooled, bridge and silo models trade cost and simplicity against isolation; mature products mix them.
  • Derive the tenant from identity and scope all data access automatically, with database-level backstops.
  • Include the tenant in caches, indexes, paths and keys.
  • Per-tenant limits, fair scheduling, placement and metrics contain noisy neighbours.

Get new lessons by email

Get new posts on backend and AI internals. No spam, unsubscribe anytime.

Your email is only used to send new posts. Privacy