Skip to content
learn
  1. Learn
  2. High Level System Design
  3. Architecture Styles

System Design HLD-14.05

Serverless and event-driven compute

Functions as a service: event triggers, scaling to zero, cold starts, limits, the cost curve versus always-on servers, and good and bad fits.

Intermediate 10 min Last reviewed

The big idea

A user uploads a photo and you need to make three thumbnails. With servers, you run a fleet sized for the busiest hour, patch its operating system, and pay for it at 4 a.m. when nobody uploads anything. With serverless functionsA cloud model where you deploy individual functions that the platform runs on demand in response to events, scaling automatically and billing only for execution time. (functions as a service), you upload just the resize function and attach it to the "file uploaded" event. The platform runs a copy for each upload, a thousand at once if a thousand photos arrive, none at all when nothing happens, and bills you per invocation and per millisecond of execution.

You stop managing servers, capacity and scaling. In exchange you accept the platform's rules: functions are short-lived and stateless, startup can add latency, and at steady high load paying per execution can cost more than simply running servers.

Think of hiring taxis instead of owning a fleet of cars. For occasional trips, taxis are cheaper and you never worry about parking, insurance or maintenance. If you drive all day every day, owning a car wins.

How it works

Step 1: Events trigger functions

Events triggering short-lived function instances Three event sources on the left: an HTTP request through an API gateway, a file uploaded to object storage and a message arriving on a queue. Each event starts a short-lived function instance, which glows while running. A burst of uploads starts several parallel instances. With no events, no instances run. Functions write results to a database and to object storage. HTTP request via API gateway File uploaded object storage Queue message fn: api handler fn: resize fn: resize fn: resize fn: send email Database Object storage Figure 1. Each event starts a function instance that runs briefly and stops. A burst of uploads runs many instances in parallel; with no events, nothing runs and nothing is billed. State lives in databases and storage, never in the function.

A function is a small piece of code with a handler that receives an event: an HTTP request through an API gateway, a new object in storage, a message on a queue or stream ( Message queues vs pub/sub vs event streams ), a database change, or a schedule. The platform keeps a pool of execution environments; each one handles one event (or a few) at a time. Concurrency equals the number of events in flight, and drops to zero when they stop.

Step 2: Cold starts

When no warm environment is available, the platform must create one: allocate a sandbox, load your code and dependencies, and run your initialization. That is a cold startThe extra latency when a serverless platform has to create a new execution environment for a function, rather than reusing a warm one., ranging from tens of milliseconds to seconds depending on runtime, package size and initialization work. Warm environments are reused for later events, so the cost shows up on traffic spikes and after idle periods. Mitigations: keep packages small, do expensive setup once outside the handler, pick fast-starting runtimes, and pay for pre-warmed capacity on latency-critical paths.

Step 3: Limits and statelessness

Functions are designed for short work: platforms cap execution time (AWS Lambda's maximum is 15 minutes), memory and payload size. Anything that must survive between invocations, such as sessions, caches or progress, must live outside, in a database, cache or object store ( Stateless vs stateful services ). Massive concurrency can also overwhelm downstream systems: a thousand parallel functions can open a thousand database connections, so put a queue or connection pooler in between ( Backpressure and flow control ).

Step 4: The cost curve

breakeven.pyruns with python3 (standard library)
# Illustrative round-number prices (not any provider's real price list):
PER_MILLION_REQUESTS = 0.20        # dollars per million invocations
PER_GB_SECOND = 0.0000167          # dollars per GB-second of function execution
SERVER_PER_MONTH = 60.0            # dollars for an always-on server that handles 200 req/s
MEM_GB, DURATION_S = 0.5, 0.1      # each request: 512 MB for 100 ms
SECONDS = 30 * 24 * 3600

print(f"{'avg load':>10} {'functions $/month':>18} {'servers $/month':>16}")
for rps in (0.1, 1, 10, 50, 200, 1000):
    reqs = rps * SECONDS
    fn = reqs / 1e6 * PER_MILLION_REQUESTS + reqs * MEM_GB * DURATION_S * PER_GB_SECOND
    servers = max(1, -(-rps // 200)) * SERVER_PER_MONTH   # at least one, always on
    print(f"{rps:>8} /s {fn:>18,.2f} {servers:>16,.2f}")

Output

  avg load  functions $/month  servers $/month
     0.1 /s               0.27            60.00
       1 /s               2.68            60.00
      10 /s              26.83            60.00
      50 /s             134.14            60.00
     200 /s             536.54            60.00
    1000 /s           2,682.72           300.00

With these illustrative round-number prices, functions cost cents at low or spiky load where a server would sit mostly idle, but cost several times more than an always-on server at steady high load. The crossover depends on real prices, memory, duration and how bursty the traffic is, so compute it for your workload.

Going deeper

Good and bad fits

Good fits: event-driven glue (process uploads, react to database changes), spiky or unpredictable traffic, scheduled jobs, low-traffic APIs and internal tools, and prototypes. Poor fits: steady high-throughput services, latency-critical paths that cannot tolerate cold starts, long-running jobs, workloads that need local state or large in-memory caches, and pipelines that pass large amounts of data between many small steps.

Serverless beyond functions

The same pay-per-use, no-servers model now covers containers that scale to zero, databases, queues and storage. The architectural question is the same: pay for what you use and give up some control, or provision capacity and run it yourself.

Trade-offs

OptionWins whenLoses when
Serverless functions Spiky, event-driven or low-volume workloads; minimal operations Cold starts, execution limits, cost at steady high load, vendor coupling
Containers on a managed platform Steady services needing control over runtime, memory and long requests Capacity planning and more operational work
Scale-to-zero containers A middle ground: container flexibility with pay-per-use Still has cold starts; platform limits apply

Common mistakes

  • Running steady, high-volume services on functions and paying several times more than servers would cost.
  • Keeping state in function memory and expecting it to persist.
  • Thousands of concurrent functions opening thousands of database connections.
  • Ignoring cold starts on user-facing latency-critical paths.
  • Chaining many small functions that pass large data through storage, paying for every hop.

In the wild

Prime Video

Scaling up the audio/video monitoring service and reducing costs by 90% (archived) (opens in a new tab)

Prime Video's 2023 post describes a stream quality monitoring tool first built from serverless components: AWS Step Functions orchestrated the work, and frames were split out of videos, stored temporarily in Amazon S3 and processed by detectors running on AWS Lambda. It hit a hard scaling limit at around 5 percent of the expected load, and the most expensive parts were the per-transition orchestration and passing data between components through S3. Moving all components into a single process running on EC2 and ECS kept data in memory and, the team reports, reduced infrastructure cost by over 90 percent.

Check yourself

  1. What does 'scale to zero' mean for serverless functions?

    Show answer

    B. When there are no events, no instances run and nothing is billed for execution Concurrency follows events, down to none when idle.

  2. Why does a function sometimes take 800 ms instead of 30 ms after a quiet period?

    Show answer

    B. A cold start: the platform had to create and initialize a new environment No warm environment was available, so setup cost was added to that request.

  3. A service handles a steady 1,000 requests per second all day. What is likely cheaper?

    Show answer

    B. Always-on servers or containers Pay-per-execution loses to provisioned capacity at steady high utilization.

  4. Where should a function keep a user's shopping cart between requests?

    Show answer

    B. In an external store such as a database or cache Function instances are ephemeral and may not be reused; state must live outside.

Recap

  • Serverless runs functions on events, scales automatically from zero and bills per execution.
  • It removes server and capacity management for spiky and event-driven work.
  • Cold starts, time and memory limits, and statelessness are the main constraints.
  • At steady high load, always-on servers are often much cheaper; compute the crossover.
  • Protect downstream systems from massive concurrency with queues and connection pooling.

Get new lessons by email

Get new posts on backend and AI internals. No spam, unsubscribe anytime.

Your email is only used to send new posts. Privacy