- Learn
- Artificial Intelligence
- Retrieval Augmented Generation
AI-M08
Retrieval Augmented Generation
How to give a model knowledge it was never trained on: ingest and chunk documents, embed and index them, retrieve and rerank, evaluate, and run it all in production.
A language model only knows what was in its training data, and it knows nothing about your contracts, tickets or last week's price change. Retrieval augmented generation fixes that by looking up the relevant passages at question time and handing them to the model with an instruction to answer from them and cite them.
You will build up the pipeline piece by piece: parsing and chunking documents, choosing an embedding model and indexing vectors, and the approximate nearest neighbour indexes that make search fast at scale. Then you will improve retrieval with hybrid keyword and vector search, reranking, contextual chunks and query rewriting, and see when a knowledge graph or a SQL database is the better source. The module ends with how to measure retrieval and answer quality separately, and with the architecture of a production system: fresh ingestion, access control, caching, citations and guardrails.
Lessons
- AI-08.01 Why RAG? Retrieval augmented generation gives a model knowledge it was not trained on by retrieving relevant passages from your own data at question time and adding them to the prompt, with instructions to answer from them and cite them. Beginner 10 min
- AI-08.02 Document ingestion and chunking Documents are split into chunks small enough to retrieve precisely but large enough to keep meaning, typically a few hundred tokens with some overlap. Intermediate 14 min
- AI-08.03 Embedding and indexing Each chunk is converted into a vector by an embedding model and stored with its text and metadata. Intermediate 10 min
- AI-08.04 Vector databases and ANN indexes Comparing a query against every vector is exact but too slow at scale, so vector databases use approximate nearest neighbour indexes. Advanced 16 min
- AI-08.05 Hybrid search and reranking Vector search finds meaning but can miss exact terms; keyword search with BM25 finds exact terms but misses paraphrases. Intermediate 12 min
- AI-08.06 Contextual retrieval and query transformation Chunks lose meaning when cut out of their document: "the limit is 10 per day" does not say which product. Advanced 12 min
- AI-08.07 GraphRAG and structured retrieval Some questions are better answered from structured data. Advanced 10 min
- AI-08.08 Evaluating RAG Evaluate retrieval and generation separately. Intermediate 12 min
- AI-08.09 Production RAG architecture Production RAG needs reliable ingestion pipelines that keep the index fresh and remove deleted documents, access control applied at retrieval time, caching for common questions, citations in the UI, monitoring of retrieval quality and guardrails that catch unsupported answers. Advanced 15 min