START HERE

What you will learn

  • Explain the difference between finding evidence and writing an answer.
  • Follow a question through a simple document-answering system.
  • Recognise why citations and access controls still need checking.

Before you start: No coding required. It helps to have read the guide to prompts or semantic search.

The basic idea: look it up before answering

Imagine asking a colleague a question about an internal policy. They find the relevant policy, read it and answer with a reference to the passage. RAG arranges a similar sequence around an AI model: search first, then generate an answer with the retrieved text in context.

Retrieval means finding candidate passages. Generation means producing an answer. Keeping these two jobs separate helps explain failures: the system might find the wrong document, or it might find the right passage and still misstate it. Improving the wording of the answer will not fix a search process that never finds the required evidence.

What RAG does and does not change

In a typical RAG application, supplying documents for a question does not update the model’s learned parameters, the values fitted during training. The retrieved passages are additional context for that answer. Updating the document collection and training the model are different operations.

RAG can make sources available and easier to inspect. It does not make every answer true. A source may be stale, incomplete or contradictory; a model may misread it. The application must also restrict retrieval to information the person is entitled to see. Asking the model to hide an unauthorised passage after retrieval is not an adequate access boundary.

Step-by-step process

  1. Prepare an approved collection

    Choose documents with clear owners and versions. Keep enough information to identify each source and decide who may read it.

  2. Find passages for the question

    Search the collection using the question. The search can use keywords, semantic search or a combination. Select useful passages without flooding the model with unrelated text.

  3. Give the model the question and passages

    Ask for an answer based on the supplied evidence. Include a clear instruction to state when the evidence does not establish an answer.

  4. Connect the answer to its sources

    Return references that let the reader open the supporting passages. Check that each important claim is actually supported by the passage it cites.

  5. Test the two jobs separately

    First ask whether the relevant passage was found. Then ask whether the answer represents it accurately. Add missing-answer and conflicting-document cases to the test set.

FOLLOW ONE EXAMPLE

Worked scenario

A question about an equipment expense

In this fictional example, a policy says equipment purchases above £200 require prior approval. A stakeholder asks whether a £250 purchase needs approval.

  1. The search retrieves the current equipment policy and the passage containing the £200 threshold.
  2. The model answers that the £250 purchase requires prior approval, and identifies the policy passage.
  3. The stakeholder then asks how long approval takes. The supplied policy gives no timescale, so the system should say the documents do not establish one.

What this shows: The first answer is supported by a specific rule. The second exposes a genuine information gap instead of inventing a service promise.

The process at a glance

The documents supply evidence for this answer. A missing fact should remain visible, rather than being filled in with a plausible guess.
ProcessDecisionHuman checkpoint
Read Mermaid diagram source
flowchart TD
 A[Ask a question] --> B[Search documents the user may access]
 B --> C[Select relevant passages]
 C --> D{Enough evidence?}
 D -->|Yes| E[Draft an answer with references]
 E --> F[Reader checks the supporting passages]
 D -->|No| G[State what is not established]

What you should have at the end

  • An answer whose key claims can be traced to accessible passages.
  • A way to distinguish search failures from answer-writing failures.
  • A clear response when the documents do not establish the answer.

Common mistakes and how to avoid them

Assuming retrieval guarantees truth

Retrieved text can be wrong or out of date. Document quality still matters.

Counting citations without opening them

A real citation can point to a passage that does not support the claim.

Indexing everything without permissions

Design document access before making the collection searchable.

APPLY THE IDEA

Try it yourself

Your task: Choose three approved documents. Write two questions they answer and one they do not answer. Identify the exact passages manually.

What to produce: A small question-and-evidence table you can use to test a future RAG prototype.

Check your understanding: For each failed answer, can you say whether the evidence was missing, not retrieved or misinterpreted?

Key terms in plain English

RAG
A pattern that combines retrieving information with generating an answer.
Passage
A selected portion of a document supplied as evidence.
Context
Information supplied to the model for the current request.
Provenance
Information showing where a claim or document came from.

Sources and further reading