RAG quality checklist

Data & chunking

• ☐ Source docs are **clean** (no broken PDF text)
• ☐ Chunk size tested (start 300–800 tokens)
• ☐ **Overlap** between chunks (~10–20%)
• ☐ Metadata stored (source, page, section title)

Retrieval

• ☐ Embedding model matches your domain/language
• ☐ Top-k tuned (usually 3–8 chunks)
• ☐ Consider **hybrid search** for SKU/error codes
• ☐ **Re-rank** if vector order is noisy
• ☐ "No relevant docs" path → refuse or ask clarifying question

Generation

• ☐ Prompt says **use only provided context**
• ☐ Citations or source labels shown to user
• ☐ Context fits **context window** after packing
• ☐ Temperature low for factual Q&A

Evaluation

• ☐ 20–50 **golden questions** with expected sources
• ☐ Track hit rate: correct chunk retrieved?
• ☐ Track faithfulness: answer supported by chunk?

Common fixes