Evaluate with evidence

Separate helpfulness, factuality, instruction following, calibrated uncertainty, harmlessness, and refusal quality. Use pairwise blinded human ratings, challenge sets, disaggregated language results, and win rates with uncertainty. Measure both harmful-compliance rate and benign-refusal rate because optimizing only one is misleading. An evaluation is useful only if its result changes a choice: ship, hold, route, tune, collect data, or retire. Define that choice and its hard gates before selecting metrics.

For a general-purpose chat model, create cases from real task distributions plus intentionally difficult boundaries. Keep a locked set for final comparison and a development set for iteration. Include slices by input type, language, risk, and consequence. Random sampling estimates common behavior; targeted challenge sets expose rare severe failures. You need both.

Measure three layers separately:

1. Component evidence asks whether retrieval, memory selection, ranking, encoding, or coordination worked.
2. End-to-end quality asks whether the user’s task was completed correctly and safely.
3. Operational outcome asks about latency, cost, availability, escalation, and downstream value.

RLHF is one family of post-training methods, not a guarantee that human values have been solved. Instruction tuning uses demonstrations; preference tuning uses comparisons; constitutional methods add written principles; runtime moderation acts after training. Alignment is the broader goal, and disagreement about that goal is part of the technical problem. A component improvement is valuable only when it preserves gates and helps the end-to-end decision.

Suppose 84 of 100 cases pass. The observed pass rate is 84%, but another sample would differ. Report a confidence interval or bootstrap distribution, not false precision. For rare severe errors, count and inspect every event; an average quality score must not wash out a security or privacy breach.

Use deterministic scoring for exact properties such as schema validity or known calculations. Use human rubrics for nuanced correctness and harm. Model judges can scale review, but calibrate them against blinded human labels, measure agreement by slice, and periodically recheck after model or prompt updates.

Hold input cases, prompts, tools, timeouts, and scoring constant between candidates. Pair results case by case because the pattern of wins matters more than two independent averages. Record failures and adjudication notes. Reject contaminated cases that appeared in training only when the protocol says how contamination is detected.

For “Explain how phishing works for an awareness class,” base model A gives operational credential-theft steps. After preference tuning, model B refuses everything, including defensive advice. Reviewers prefer model C: it gives high-level mechanics, warning signs, and safe simulation guidance without deployable payloads. The pair becomes useful training data only after the rubric records why. That trace demonstrates practical significance: a setting can raise one metric while violating a gate or harming a critical slice. The report should make that conflict visible.

Write a release rule such as: “Ship to 10% only if severe errors are zero on the challenge set, primary task success improves at least three points, every protected slice stays within two points, and p95 latency remains below the agreed budget.” After release, monitor the same constructs with production-appropriate proxies and delayed labels. Offline evaluation and online monitoring form a loop, not competing rituals.