Skip to content
learn
  1. Learn
  2. High Level System Design
  3. Asynchronous Processing and Messaging

System Design HLD-09.09

Event-driven architecture and event sourcing

Events vs commands, services that react to facts, and event sourcing: state as a fold over an event log, with snapshots, projections and real pitfalls.

Advanced 14 min Last reviewed

The big idea

There are two ideas in this lesson that sound alike and are often confused.

Event-driven architecture is about how services talk. Instead of the order service calling billing, then shipping, then email, it announces a fact, "order 42 was placed", and every service that cares reacts on its own. The order service does not know or wait for them.

Event sourcing is about how a service stores its own data. Instead of keeping only the current state, "balance: 528", it keeps every change that ever happened, "opened, deposited 100, deposited 250, withdrew 80, ...", and computes the current state by replaying them. The log of events is the database.

Think of a bank statement. Your balance is not a number someone types in; it is the result of every transaction on the statement, in order. Lose the balance and you can recompute it. Want to know your balance last Tuesday, or how much you spent on fees this year? It is all in the history.

How it works

Step 1: Events vs commands

A commandA request asking a specific service to do something, which it may accept or reject, such as 'PlaceOrder' or 'ChargeCard'. asks someone to do something: ChargeCard. It is addressed to one receiver, it can be rejected, and the sender usually cares about the outcome. An eventA record of something that has already happened, named in the past tense, such as 'OrderPlaced'; it cannot be rejected, only reacted to. states a fact that already happened: CardCharged. It is broadcast, it cannot be rejected, and the sender does not care who listens.

The naming matters because it shapes coupling. A service that publishes OrderPlaced can gain three new consumers without a single code change. A service that sends SendShippingRequest to shipping has to know shipping exists and what it needs.

Step 2: Event-driven architecture

Services react to each other's events via a broker ( Message queues vs pub/sub vs event streams ). Typical flows are choreography: order publishes OrderPlaced, payment reacts and publishes PaymentCaptured, shipping reacts to that, and so on, with no central coordinator. You gain loose coupling and independent scaling. You give up a single place that shows the whole flow, and consistency becomes eventual: for a moment the order exists but shipping does not know yet. Long multi-step business flows across services, with compensation when a step fails, are sagas ( Sagas: choreography and orchestration ).

Step 3: Event sourcing: state is a fold over events

Rebuilding an account balance from its event log Five events in a row: AccountOpened, Deposited 100, Deposited 250, Withdrawn 80, FeeCharged 5. Under each, the balance after applying it: 0, 100, 350, 270, 265. A glowing cursor moves from left to right as each event is applied. Opened Deposited 100 Deposited 250 Withdrawn 80 Fee 5 0 balance 100 balance 350 balance 270 balance 265 balance Figure 1. The balance is never stored and updated in place. It is computed by applying each event in order: 0, 100, 350, 270, 265. Step through the replay.
  1. AccountOpened Start from the empty state. The account exists with balance 0.
  2. Deposited 100 Apply the deposit: 0 + 100 = 100.
  3. Deposited 250 100 + 250 = 350.
  4. Withdrawn 80 350 - 80 = 270.
  5. FeeCharged 5 270 - 5 = 265. That is the current balance: a fold over the whole log.

In event sourcingStoring every change to an entity as an immutable event in an append-only log, and deriving current state by replaying those events., each entity (an account, an order) has its own append-only stream of events. To handle a command:

  1. Load the entity's events and replay them to get its current state.
  2. Check the command against that state: is there enough balance for this withdrawal?
  3. If valid, append one or more new events. Never update or delete old ones.

The append is protected by an expected version: "append Withdrawn 80 only if this stream is still at version 3". If someone else appended first, the write fails and the command is retried against the fresh state. That is optimistic concurrency, and it is what stops two withdrawals from both spending the same money.

Step 4: Projections

Replaying a stream answers "what is account 7's balance?", but not "which accounts are overdrawn?". For queries, the events are also fed to projectionsA read model built by processing events into a shape suited to specific queries, such as a table of balances or a search index; it can be deleted and rebuilt from the log.: read models, such as a balances table, a search index or a monthly report, each updated as events arrive. This is CQRS ( Denormalization, materialized views and CQRS ) with the event log as the write side.

The superpower: a projection is disposable. Need a new view next year, such as "all withdrawals over 100 for the fraud team"? Write the projection and replay the entire history into it. The data was there all along.

Step 5: Snapshots

An account with 50,000 events is slow to replay on every command. A snapshotA saved copy of an entity's state as of a specific event version, so replay can start there instead of from the first event. stores the state at some version, say every 1,000 events. To load the entity, read the latest snapshot and replay only the events after it. Snapshots are a cache: delete them and you can rebuild them from the log.

Try it

This interactive needs JavaScript. The diagrams and text above cover the same idea.

Try this:

  1. Press Next a few times with the balance projection. Each step applies one event; nothing is overwritten. Press Back: the state is simply recomputed with one fewer event.
  2. Play to the end, then switch to new: large withdrawals. A view that never existed before is built instantly from the old events.
  3. Back on the balance projection, turn on the snapshot and play to the end. Count the events replayed: only those after the snapshot.

Going deeper

Pitfalls

  • Schema evolution: events are kept forever, so code must read every version of every event ever written. Teams version event types and write "upcasters" that convert old shapes to new ones on read.
  • Eventual consistency of projections: a user who just deposited money may briefly see the old balance in a projection. Read from the stream for the user's own writes, or show pending state.
  • Deleting data: "never delete" collides with privacy law. Common approaches are keeping personal data out of events, or encrypting it with a per-person key and deleting the key (crypto-shredding).
  • Complexity: event sourcing is a big commitment. It pays off in domains that are naturally a history: ledgers, orders, bookings, workflows, audit-heavy systems. For a simple CRUD screen it is overkill.

Event sourcing is not the same as Kafka

A broker topic is great for distributing events between services, but event sourcing needs to load all events for one entity quickly and append with an expected-version check. Teams usually store event streams in a database built or configured for that, then publish them to a broker for other services, often with the outbox pattern ( Transactional outbox and change data capture ).

Trade-offs

OptionWins whenLoses when
State-based storage (CRUD) Most applications; simple queries and updates History, audit and the ability to answer new questions about the past
Event sourcing Ledgers, orders, workflows; full audit trail; new projections from old data Schema evolution, eventual consistency, learning curve
Event-driven integration (choreography) Loosely coupled services that react independently No single view of the end-to-end flow; harder debugging
Commands with orchestration Flows that need one owner and clear error handling Tighter coupling to the orchestrator

Common mistakes

  • Naming events as commands (SendEmail) and turning pub/sub into hidden remote procedure calls.
  • Editing or deleting past events to fix a bug instead of appending a correcting event.
  • Appending without an expected-version check, letting concurrent commands violate invariants.
  • Event sourcing an entire system, including simple CRUD parts that gain nothing from history.
  • Ignoring event versioning until old events can no longer be read.

In the wild

Accounting has worked this way for centuries: a ledger is an append-only list of entries and balances are derived from it. Git is a close cousin, storing a history of commits from which any version of the files can be rebuilt. In software, event sourcing is common in banking, trading (an exchange's order matching engine rebuilds every order book from its sequenced journal), booking and order management systems, often with frameworks such as Axon for Java or dedicated event store databases, with events then published to a broker such as Kafka for other services.

Check yourself

  1. Which is an event rather than a command?

    Show answer

    C. PaymentCaptured Events are past-tense facts that already happened; the others ask someone to do something.

  2. In an event-sourced account, how is the current balance obtained?

    Show answer

    B. Replay the account's events (from the latest snapshot) and apply each one State is a fold over the event stream; snapshots just shorten the replay.

  3. Two withdrawals for the same account arrive at once. What prevents both from spending the same money?

    Show answer

    B. An expected-version check on append: the second fails and retries against fresh state Optimistic concurrency on the stream version makes one of the two appends fail.

  4. The fraud team wants a report of every large withdrawal since the system launched. With event sourcing you...

    Show answer

    B. Write a new projection and replay the full history into it All past events are retained, so a new read model can be built from them at any time.

Recap

  • Commands ask for something and can be rejected; events state past facts and are broadcast.
  • Event-driven architecture lets services react to each other's events without direct coupling.
  • Event sourcing stores every change as an immutable event; state is a replay of the stream.
  • Projections are disposable read models, so new views can be built from old events; snapshots speed up replay.
  • Costs: schema evolution, eventual consistency, deletion and complexity. Use it where history is the domain.

Get new lessons by email

Get new posts on backend and AI internals. No spam, unsubscribe anytime.

Your email is only used to send new posts. Privacy