Reference · Glossary
A/B testing
Splitting traffic between **two variants** (prompt, model, retrieval) and comparing metrics to pick a winner.
When to use
Prompt or model changes where offline eval is inconclusive.
When not to
When sample size is too small — noise beats signal.
Example
50% users get prompt v1, 50% v2 — compare groundedness over a week before full rollout.