Skip to content
learn
  1. Learn
  2. High Level System Design
  3. Infrastructure Essentials

System Design HLD-17.01

Containers and Kubernetes for system designers

Images, containers, pods, deployments and services; how Kubernetes reconciles declared state, heals failures, and what it does not solve for you.

Intermediate 15 min Last reviewed

The big idea

"It works on my machine" was once the most common sentence in software. A containerA packaged, isolated process that bundles an application with its libraries and settings, so it runs the same way on any machine with a container runtime. fixes it by packaging an application together with everything it needs, its libraries, runtime and settings, into an imageAn immutable, versioned package from which containers are started. Built once, run anywhere.. The same image runs identically on a laptop, a test server and a production cluster.

Once applications are containers, something has to run hundreds or thousands of them across a fleet of machines: decide where each goes, restart the ones that crash, move them when a machine dies, roll out new versions and route traffic to whichever copies are healthy. That is what KubernetesAn open source system that runs containers across a cluster of machines, continuously reconciling the actual state toward a declared desired state. does. Its central idea is simple and powerful: you declare what you want, and controllers keep working until reality matches.

Think of a thermostat. You do not tell the heater when to switch on; you declare "21 degrees", and a loop keeps comparing the room with the target and acting. Kubernetes runs that kind of loop for your whole infrastructure.

How it works

Step 1: The building blocks

  • Pod: the smallest unit Kubernetes runs, one or a few tightly coupled containers sharing a network address. Pods are disposable: they are created and destroyed, never repaired.
  • Deployment: "keep 3 replicas of this pod template running, at this image version". It handles rolling updates and rollbacks.
  • Service: a stable name and virtual IP that load-balances across whichever pods currently match, so clients never track individual pods (service discovery, Service discovery ).
  • Node: a machine (virtual or physical) that runs pods. The scheduler picks a node for each new pod based on its CPU and memory requests and placement rules.

Step 2: The reconciliation loop

The Kubernetes reconciliation loop A ring of four stages: you declare the desired state, for example three replicas; a controller observes the actual state; it compares the two and finds a difference; it acts by creating or deleting pods; then it observes again. The loop runs continuously. Declare replicas: 3, image v2 Observe only 2 pods running Diff 1 pod missing Act create a pod desired = actual? forever Figure 1. Controllers never stop comparing desired with actual state. A crashed pod, a lost node or a new version all become the same thing: a difference to close.

You never say "start a container on node 7". You submit desired state to the API server, which stores it in etcd (a consensus-backed store, Coordination services (ZooKeeper, etcd) ). Controllers watch for differences and act. If a node dies, its pods simply stop being counted, the controller sees too few replicas, and the scheduler places new pods on healthy nodes. Self-healing is not a special feature; it falls out of reconciliation.

Kubernetes control plane and worker nodes The control plane holds the API server, etcd, the scheduler and controllers. Worker nodes run a kubelet and pods. A Service load balances traffic to the pods of a deployment spread across nodes. One node fails and its pod is recreated on another node. Control plane reschedule API server etcd scheduler + controllers Node 1 pod A Node 2 failed Node 3 pod B (new) Service stable name Figure 2. The control plane stores desired state and runs the controllers; kubelets on each node start the pods assigned to them. When node 2 fails, its pod is recreated on node 3 (glowing) and the Service routes around the gap.

Step 3: What Kubernetes solves, and what it does not

It solves deployment mechanics: packaging, placement, restarts, rolling updates, service discovery, configuration and horizontal scaling of stateless services ( Autoscaling ).

It does not solve architecture. Your database still needs replication, backups and a sharding plan; your services still need timeouts, retries and idempotency; a bad design is just as bad inside a cluster. Running stateful systems like databases on Kubernetes is possible with StatefulSets and operators, but it is harder, and many teams use managed databases instead. Kubernetes itself is also a complex system to operate, which is why most teams use a managed offering.

Try it

Does Kubernetes handle this for you, or is it still your design problem?

Show answers
  • Restarting a service process that crashed: Kubernetes handles it. The deployment controller sees a missing replica and recreates the pod.
  • Rolling out version 2 one pod at a time: Kubernetes handles it. Deployments do rolling updates with readiness checks and can roll back.
  • Choosing a shard key for the orders database: Still your design. Data partitioning is architecture; Kubernetes knows nothing about your data.
  • Moving pods off a machine that died: Kubernetes handles it. Reconciliation reschedules them onto healthy nodes.
  • Making payment retries safe: Still your design. Idempotency is application logic.
  • Deciding how stale cached prices may be: Still your design. Consistency requirements come from the product, not the platform.

Trade-offs

OptionWins whenLoses when
Managed Kubernetes Many services and teams that need a common platform for deploys, scaling and healing Learning curve and operational complexity for small teams
Simpler platforms (PaaS, serverless) Small teams and few services: less to run Fine control over networking, placement and custom workloads
Databases on Kubernetes Teams with strong operators and tooling who want one platform Simplicity and safety compared with managed databases

Common mistakes

  • Treating pods as pets: logging in to fix one instead of fixing the image and redeploying.
  • No resource requests or limits, so the scheduler packs nodes badly and one pod starves others.
  • Missing readiness probes, so traffic reaches pods before they can serve.
  • Expecting Kubernetes to fix architectural problems such as a single overloaded database.

Check yourself

  1. What does a Kubernetes Deployment declare?

    Show answer

    B. A desired number of pod replicas from a template and image version Controllers keep that many replicas of the given version running.

  2. A node dies. How do its pods come back?

    Show answer

    B. Controllers see too few replicas and the scheduler places new pods on healthy nodes Self-healing is reconciliation: the gap between desired and actual is closed automatically.

  3. Why do clients talk to a Service instead of to pods?

    Show answer

    B. Pods come and go; the Service is a stable name that load-balances to current healthy pods Pods are disposable, so a stable abstraction routes to whichever exist.

  4. Which problem does Kubernetes not solve for you?

    Show answer

    C. Partitioning and replicating your database correctly Data architecture remains your design responsibility.

Recap

  • Images package an app with its dependencies; containers run it the same way everywhere.
  • Pods, deployments, services and nodes are the core Kubernetes building blocks.
  • You declare desired state; controllers reconcile reality toward it, which gives self-healing.
  • Services give disposable pods a stable name and load balancing.
  • Kubernetes handles deployment mechanics, not your data architecture.

Get new lessons by email

Get new posts on backend and AI internals. No spam, unsubscribe anytime.

Your email is only used to send new posts. Privacy