Reference · Glossary
RAG
Last updated
- Answers can cite sources you control (policies, product docs, tickets).
#Why teams use it
- Answers can cite sources you control (policies, product docs, tickets).
- Knowledge stays fresher than waiting for a model retrain.
- You can inspect which chunks were retrieved when something goes wrong.
#Pipeline (mental model)
1. **Ingest** — split docs into chunks and store embeddings in a vector index (or hybrid keyword + vector).
2. **Retrieve** — embed the user question, fetch top-k related chunks.
3. **Generate** — prompt the LLM with those chunks and instructions to stay faithful.
4. **Evaluate** — check faithfulness, citation coverage, and refusal when evidence is missing.
#When to use
You need answers grounded in your notes, docs, or product knowledge — not only the model’s training cut-off. Classic fits: internal help desks, research assistants over a paper set, and product Q&A.
#When not to
Tiny one-off questions with no private corpus, or when a single pasted paragraph already fits in the prompt. Also skip RAG if your real problem is style/format consistency — that often points to prompting or fine-tuning instead. See RAG vs fine-tuning.
#Failure modes to watch
- **Wrong chunks** — retrieval misses the right section; fix chunking, metadata filters, or hybrid search.
- **Ignoring context** — model invents anyway; tighten the prompt and add “say you don’t know.”
- **Stale index** — docs changed but the vector store did not; schedule reindex jobs.
#Example
Retrieve: top 3 chunks about refund policy
Generate: Answer using only those chunks. If missing, say you don't know.
Cite chunk ids in the reply.#Learn next
- Interactive lesson: `what-is-rag`
- How-to: Build a 5-doc RAG
- Glossary: embedding, vector database