System Design HLD-06.05
NoSQL families
Key-value, document, wide-column, graph, time-series and search: the data model and query patterns each family is built for, with real examples.
The big idea
"NoSQL" is not one technology; it is a label for databases that give up the general-purpose relational model in exchange for being exceptionally good at one shape of data or one kind of query. A relational database can store almost anything and answer almost any question reasonably well. A NoSQL database answers its kind of question extremely well, often at a scale or speed a single relational server cannot reach, and handles other questions poorly or not at all.
So the useful question is never "SQL or NoSQL?" but "what shape is my data, and which queries must be fast?" Each family below is the answer to a particular pair. Many real systems use several: a relational database for orders, a key-value store for sessions, a search engine for the product catalog and a time-series database for metrics.
Think of kitchen tools. A chef's knife does almost everything well. A cherry pitter, a mandoline and a garlic press each do one job far better and are useless for anything else.
How it works
Step 1: The families
- Key-value (DynamoDB, Redis, etcd): get, put and delete a value by its key. The simplest model and the easiest to scale; queries by anything other than the key need extra indexes or are impossible. Sessions, carts, feature flags, caches.
- Document (MongoDB, Firestore, Couchbase): each record is a self-contained JSON-like document, possibly nested, with a flexible schema and indexes on fields. Great when you read and write a whole object together (a product with its variants, a user profile); awkward when data is shared across many documents and must stay consistent.
- Wide-column (Cassandra, ScyllaDB, Bigtable, HBase): rows grouped into partitions by a partition key and sorted within the partition by a clustering key. Built for enormous write volumes spread across many nodes. You design one table per query: "messages in channel X, newest first" is a single partition read.
- Graph (Neo4j, Neptune): nodes and edges with properties. Multi-hop questions such as "friends of friends who like jazz" or fraud rings are natural, where SQL would need many expensive self-joins.
- Time-series (InfluxDB, TimescaleDB, Prometheus): timestamped measurements, compressed heavily and queried by time windows with downsampling and retention policies. Metrics, sensors, financial ticks.
- Search engines (Elasticsearch, OpenSearch, Solr): inverted indexes from words to documents, with relevance ranking, fuzzy matching and facets. Usually a secondary index built from a primary database, covered in Inverted indexes and full-text search .
Step 2: What you give up
Most NoSQL stores trade away some of: joins (you denormalize or join in application code), multi-record transactions (some now offer them within limits), ad hoc queries (access patterns must be known up front), and sometimes strong consistency by default. In return they offer horizontal scaling built in, flexible schemas or a data model that matches the problem. These are design trade-offs, not quality judgments, and the line keeps moving: relational databases added JSON columns, and many NoSQL databases added transactions.
Try it
Going deeper
Query-first modeling
With relational databases you model the data and query it however you like later. With key-value and wide-column stores you model the queries: list every access pattern first, then design keys and tables so each one is a single lookup or partition read, duplicating data where needed. Getting this wrong is expensive to fix, so these stores reward knowing your access patterns up front. The scaling side of this appears in Partitioning and sharding .
Trade-offs
| Option | Wins when | Loses when |
|---|---|---|
| Key-value | Simple lookups by key at very large scale with predictable latency | Any query not by key |
| Document | Aggregates read and written as a unit, evolving schemas | Data shared across documents that must stay consistent |
| Wide-column | Massive writes and per-partition range reads | Ad hoc queries, joins and access patterns not designed in advance |
| Graph | Deep relationship traversals | Bulk aggregation and very high write throughput |
Common mistakes
- Choosing a NoSQL store for scale the product will never have, and losing joins and transactions it needed.
- Using a document store for highly relational data, then rebuilding joins in application code.
- Designing a Cassandra table like a SQL table and querying it by non-key columns.
- Treating a search engine as the source of truth instead of an index of one.
Check yourself
-
Which family is built for 'friends of friends who also like jazz'?
Show answer
C. Graph Multi-hop traversal of relationships is what graph databases do best.
-
What is the main design rule for wide-column stores like Cassandra?
Show answer
B. Design tables around the queries, partitioned by key Each query should hit one partition, so tables are designed per access pattern.
-
Why is a search engine usually not the source of truth?
Show answer
B. It is an index built for search, typically populated from a primary database The primary database holds authoritative data; the search index is derived from it.
-
A product catalog where each item has different nested attributes fits which family best?
Show answer
B. Document Documents hold nested, varying attributes and are read and written as a whole.
Recap
- NoSQL is several families, each optimized for one data shape and set of queries.
- Key-value for lookups by key; document for nested aggregates; wide-column for massive writes.
- Graph for relationships; time-series for metrics; search engines for full-text.
- They trade joins, transactions or ad hoc queries for scale or a better-fitting model.
- Choose by access patterns, and expect to combine several stores.