Retrieval-Augmented Generation
Also called: RAG, retrieval augmented generation
Fetching relevant information from your own data at question time and giving it to the model, instead of relying on what the model already knows.
Instead of hoping a model already knows about your refund policy, you search your own documents for the relevant passages and include them in the request. The model answers from what you supplied.
This is the standard architecture for almost every business AI application, for three reasons: your data stays yours, answers can cite their sources, and updating knowledge means updating a document rather than retraining anything.
What determines whether it works
Answer quality is capped by retrieval quality. If the search step returns the wrong three paragraphs, no model can produce the right answer from them.
This is why most "the model is bad" complaints are actually retrieval problems. Naive chunking plus cosine similarity works impressively in a demo with fifty documents and degrades badly at fifty thousand. Hybrid keyword-plus-vector search, reranking and metadata filters are what close that gap.
What people get wrong
Treating retrieval as a solved step because a tutorial made it look like three lines of code. Chunk size, chunk overlap, what metadata you attach, whether you rerank, and how you filter by the user's permissions are all consequential decisions with measurable effects. Measure them with an evaluation set.
Related terms