Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
TL;DR
Combine a parametric language model (BART) with a non-parametric retriever (DPR) over Wikipedia. The model reads relevant passages before answering — so it gets facts right without storing them all in weights, and you can cite sources.
Why it matters
Founded RAG as a paradigm. The reason every modern enterprise AI stack starts with 'pick a vector database'. Direct ancestor of ChatGPT-with-citations, Perplexity, Glean, NotebookLM, and every doc-QA app. The dominant approach for grounding LLMs in private data.
The Key Idea
Don't store all knowledge in the model's weights. Retrieve relevant passages from an external corpus at inference time and condition generation on them. The knowledge lives outside the model, can be updated freely, and supports citations.
How it works
1. Dense Passage Retriever (DPR)
A bi-encoder: two BERT models, one encoding the query and one encoding each passage in the corpus. Train them with a contrastive loss so query embeddings cluster near relevant passage embeddings. At inference, FAISS does approximate nearest-neighbor search over the precomputed passage embeddings (sub-100ms for tens of millions of passages).
2. Seq2Seq Generator (BART)
A pretrained encoder-decoder Transformer. Input: the query concatenated with the top-k retrieved passages. Output: the generated answer. BART was chosen because it was pretrained on denoising — well-suited to consuming noisy or partially-relevant context.
3. Two Variants: RAG-Sequence vs RAG-Token
RAG-Sequence: pick the top-k passages once, generate the entire answer conditioned on each, marginalize. RAG-Token: for every generated token, marginalize over the passages — different tokens can effectively 'attend to' different passages. RAG-Token costs more but is more flexible.
4. End-to-End Differentiable Training
Both the retriever and generator are trained jointly. The generation loss flows back through the retrieval scores (via a marginalization trick), so the retriever learns to surface passages the generator can actually use. This is rare — most modern RAG systems train retriever and generator separately.
5. Corpus: Wikipedia + DPR Embeddings
Wikipedia split into ~21M passages of 100 words each, all pre-embedded with DPR. At inference, the entire corpus is just a FAISS index sitting on disk. Update Wikipedia → rebuild index → no model retraining required.
Results
- ✓NaturalQuestions: 44.5 EM — state of the art on open-domain QA
- ✓TriviaQA: 56.8 EM — significantly better than closed-book LMs
- ✓Fact-verification (FEVER): 72.5% label accuracy — beat fine-tuned LMs
- ✓Generated answers were more factual and less hallucinated than parametric-only baselines
- ✓Passage updates propagated immediately — no retraining required to teach new facts
Takeaways
- →Retrieval beats parametric storage for facts. The model focuses on language, the index focuses on knowledge.
- →Knowledge is updatable. Re-index, no retraining.
- →Citations are free. You retrieved the passage — show it.
- →Founded modern RAG. Everything since (CRAG, GraphRAG, Adaptive, Multimodal) is a variant of this template.