RAG and Retrieval Systems

Introduction to Retrieval-Augmented Generation

Retrieval-augmented generation, or RAG, finds relevant information from an external collection and gives that evidence to a language model before it writes an answer.

A language model can produce a confident sentence by repeatedly selecting likely next tokens. That process can create fluent text even when the model has no reliable evidence for the claim it is making.

For example, ask about an internal policy that was never in the model’s training data. The model may still complete the answer in a plausible style. Fluency tells us that the continuation reads well; it does not tell us that the policy is correct.

Retrieval-augmented generation addresses this problem by finding relevant source passages before the answer is written. We can then evaluate two separate questions: did retrieval find the needed evidence, and did the generated answer use that evidence correctly?