Retrieval-augmented generation (RAG)
Retrieval-augmented generation is the pattern where an LLM is given relevant context retrieved from an external source — typically via semantic search over a vector database — before generating its response. RAG grounds the model in current, authoritative content the model wasn't trained on, dramatically reducing hallucination on factual queries.
RAG emerged as the practical answer to two LLM limitations: knowledge cutoff (the model doesn't know facts from after training) and hallucination (the model invents plausible-but-wrong answers when asked about facts it doesn't have). The pipeline: chunk source documents into small passages, embed each chunk into a vector, store in a vector database, embed the user query into the same space at runtime, retrieve top-K nearest chunks, and supply them as context to the model. Production RAG includes substantial engineering around chunking strategy, retrieval re-ranking, citation extraction, and answer validation. Agentic RAG (2024+) lets the model iteratively refine queries and retrieve more context as it reasons.
Related terms
- Embedding vector
An embedding vector is a numerical representation of a piece of content (text, image, audio) as a fixed-length vector — typically 384 to 3072 dimensions — produced by a neural network trained so that semantically similar inputs produce vectors close to each other in the embedding space.
- Vector database
A vector database stores embedding vectors and supports fast approximate-nearest-neighbour search at scale — answering 'find the K vectors most similar to this query vector' in milliseconds across millions or billions of vectors.
- Hallucination
Hallucination is the LLM failure mode in which the model generates content that is plausible-sounding but factually wrong — invented citations, fabricated quotes, non-existent functions, misremembered statistics.