Pick an eval or observability tool

Match the job to the tool before wiring one in — most teams end up using two or three together, not one for everything.

| Need | Start here | Why |

|------|------------|-----|

| Score RAG answer quality (faithfulness, relevance) | RAGAS | Built specifically for retrieve-then-generate metrics |

| LLM checks inside your normal test suite | DeepEval | Feels like pytest; runs in CI |

| Compare prompts/models side by side, quick red-team checks | promptfoo | Config-driven, no code required to start |

| See exactly what happened in a multi-step chain/agent | LangSmith or Langfuse | Full trace tree per request |

| Keep observability data in your own infra | Langfuse | Open-source, self-hostable |

| Custom scoring logic no tool covers | Hand-rolled eval gate | See `eval-in-code` lesson pattern |

1. **Do you have a RAG pipeline specifically?** → add RAGAS metrics first

2. **Do you want evals living next to unit tests in CI?** → DeepEval

3. **Do you just need a quick prompt A/B before shipping?** → promptfoo

4. **Is debugging "what actually happened in this run" the pain?** → tracing (LangSmith/Langfuse), not an eval score

5. **Need it self-hosted for data residency?** → Langfuse over LangSmith

Tracing tools bill by trace volume — sample production traffic (e.g. 10%) once volume is high, and keep 100% tracing only for staging/canary.

**Related lessons:** `eval-and-benchmarks`, `eval-in-code`, `rag-quality-audit`, `llm-tracing-lab`