Reference · How-to · ~7 min
How to ship with a canary deploy
Last updated
Send a small slice of traffic to the new model or prompt before full rollout.
Send a small slice of traffic to the new model or prompt before full rollout.
Steps
1. **Pin versions** — Old and new model IDs documented.
2. **Route 5–10% traffic** — Feature flag or load balancer weight.
3. **Watch metrics** — Latency, error rate, groundedness, cost per request.
4. **Compare side-by-side** — Same golden questions on old vs new.
5. **Promote or rollback** — One-click revert if quality drops.
Copy-paste runbook
Canary start: 10% → new prompt v2
Alert if: error_rate > 2% OR p95_latency > 4s OR groundedness < 0.85
Hold 24h business hours
Promote to 100% if green; else rollback flagWatch out
Canary traffic on **low volume** can look fine while edge cases fail — include hard tests from eval suites.