A/B test lab
**Split traffic and measure quality** — prompt changes, retrievers, and models need evidence before full rollout.
Reviewed July 2026 · interactive lab
1Try it yourself
Playground
A/B test lab
Split traffic and measure quality — not every change needs an experiment, but behavior shifts do.
New system prompt — 50/50 traffic
2Read & reflect
Recap
#What you just did
Teach
#Pattern
# Pseudocode — assign variant at request time, log metrics
variant = assign_ab(user_id, experiment="system-prompt-v3")
response = run_pipeline(variant, user_message)
log_event(experiment="system-prompt-v3", variant=variant, grounded=score)
Pair with eval gates: promote only when winner beats control on your golden tasks.
Use it
#When you'd use this
- New [system prompt](/reference/system-prompt) or tool policy
- Swapping embedding model or re-ranker
- Reasoning vs fast model routing
Watch out
#Watch out
Under-powered tests lie. Need enough traffic and pre-registered metrics.
Try next
#Try this next
Pick one metric (faithfulness, p95 latency, thumbs-up rate) for your next prompt change.
3Spark check