Skip to content
learn
  1. Learn
  2. High Level System Design
  3. Infrastructure Essentials

System Design HLD-17.03

Infrastructure as code and environments

Describing servers, networks and databases in versioned files, immutable infrastructure, and keeping development, staging and production alike.

Beginner 8 min Last reviewed

The big idea

Someone once set up the production database by clicking through a cloud console. Two years later nobody knows its exact settings, the staging copy is subtly different, and rebuilding it after a disaster would be guesswork. Infrastructure as codeDescribing infrastructure such as networks, servers, databases and permissions in versioned text files that a tool applies, instead of configuring it by hand. (IaC) prevents this: networks, servers, databases, load balancers and permissions are described in text files, stored in version control, reviewed like any other code, and applied by a tool such as Terraform, OpenTofu, Pulumi or CloudFormation.

The payoff is reproducibility. A new environment is one command. Every change has an author, a review and a history. A region lost in a disaster can be rebuilt from the same files ( Disaster recovery: RPO and RTO ). And because the files describe desired state, the tool can show you exactly what will change before it changes anything.

Think of a recipe versus a dish someone improvised. Anyone can cook the recipe again and get the same result; nobody can reproduce the improvised dish.

How it works

Step 1: Declare, plan, apply

The infrastructure as code workflow An engineer edits infrastructure files and opens a pull request. The tool computes a plan by comparing the files with the real infrastructure, and the plan is reviewed alongside the code. After approval, apply makes the changes in the cloud, and the tool records the new state. approved Edit .tf files Pull request code review Plan + add ~ change - destroy Apply Cloud Figure 1. Changes start as code. The plan shows exactly what will be created, changed or destroyed; humans review it; apply makes reality match the files.

The tool compares what the files declare with what actually exists and produces a plan: what it will create, change and destroy.

plan.pyruns with python3 (standard library)
# A toy "terraform plan": compare the declared infrastructure with what exists, list the changes.
declared = {
    "vpc/main":        {"cidr": "10.0.0.0/16"},
    "db/orders":       {"engine": "postgres", "size": "large", "replicas": 2},
    "app/api":         {"image": "api:1.8.2", "instances": 4},
    "bucket/uploads":  {"versioning": True},
}
actual = {
    "vpc/main":        {"cidr": "10.0.0.0/16"},
    "db/orders":       {"engine": "postgres", "size": "medium", "replicas": 2},
    "app/api":         {"image": "api:1.8.1", "instances": 4},
    "app/legacy-cron": {"image": "cron:0.3", "instances": 1},   # created by hand, never declared
}

def plan(declared, actual):
    for name in sorted(declared.keys() | actual.keys()):
        want, have = declared.get(name), actual.get(name)
        if have is None:
            yield f"+ create  {name} {want}"
        elif want is None:
            yield f"- destroy {name} (not in code)"
        elif want != have:
            diff = {k: f"{have.get(k)} -> {v}" for k, v in want.items() if have.get(k) != v}
            yield f"~ update  {name} {diff}"

changes = list(plan(declared, actual))
print("\n".join(changes))
print(f"Plan: {sum(c[0] == '+' for c in changes)} to add, "
      f"{sum(c[0] == '~' for c in changes)} to change, {sum(c[0] == '-' for c in changes)} to destroy.")

Output

~ update  app/api {'image': 'api:1.8.1 -> api:1.8.2'}
- destroy app/legacy-cron (not in code)
+ create  bucket/uploads {'versioning': True}
~ update  db/orders {'size': 'medium -> large'}
Plan: 1 to add, 2 to change, 1 to destroy.

The plan catches two kinds of drift at once: the declared upgrade of the database size and app image, and a legacy-cron server someone created by hand that is not in the code, which apply would destroy. Reading the plan before applying is the most important habit in IaC: a one-line edit can, through dependencies, replace a database.

Step 2: Immutable infrastructure

Instead of logging in to patch running servers, build a new image with the change and replace the old servers with new ones. Immutable infrastructureNever modifying running servers; every change ships as a new image or instance that replaces the old one. means every server matches a known, tested image, rollback is redeploying the previous image, and there is no slowly accumulating hand-made difference between machines. Containers and Kubernetes ( Containers and Kubernetes for system designers ) work exactly this way.

Step 3: Environments that match

Development, staging and production should be built from the same code with different parameters (size, count, domain names), not maintained separately. The closer staging is to production, in versions, configuration and topology, the more problems it catches before users do. Differences that cannot be avoided, such as data volume, should be known and written down.

Going deeper

State and secrets

Tools like Terraform keep a state file recording what they manage. It must be stored remotely, locked during applies so two people cannot apply at once, and protected, because it can contain sensitive values. Secrets themselves belong in a secret manager, referenced by the code, never written into it ( Encryption, keys and secrets ).

Trade-offs

OptionWins whenLoses when
Infrastructure as code Anything that must be reproducible, reviewed and rebuilt: production, staging, disaster recovery Speed for one-off experiments, and a learning curve
Click-ops (console changes) Quick exploration in a throwaway sandbox Reproducibility, review and history
Immutable replacement Consistent fleets and easy rollback Image build time for every small change

Common mistakes

  • Applying without reading the plan, and destroying something by accident.
  • Making "quick fixes" in the console that drift from the code and are wiped by the next apply.
  • Committing secrets or the state file to the repository.
  • Staging built by hand and drifting until it no longer predicts production.

Check yourself

  1. What does a plan show?

    Show answer

    B. The changes apply would make: create, update and destroy It compares declared and actual state so changes can be reviewed before they happen.

  2. A server was created by hand and is not in the code. What will apply do?

    Show answer

    B. Propose to destroy it, since the code does not declare it Desired-state tools converge on what the files declare.

  3. What is immutable infrastructure?

    Show answer

    B. Replacing servers with new images instead of patching them in place Every change ships as a new image, which keeps fleets consistent and rollbacks simple.

  4. Why build staging from the same code as production?

    Show answer

    B. So it matches production closely enough to catch problems first Hand-maintained staging drifts and stops predicting production behavior.

Recap

  • Infrastructure as code describes infrastructure in versioned, reviewed files.
  • Plan shows the changes before apply makes them; always read it.
  • Immutable infrastructure replaces servers instead of patching them.
  • Build every environment from the same code with different parameters.
  • Keep state locked and protected, and secrets in a secret manager.

Get new lessons by email

Get new posts on backend and AI internals. No spam, unsubscribe anytime.

Your email is only used to send new posts. Privacy