Eval for everyone
Decide who owns the quality bar
Metrics inform a release decision; a named human owns what counts as acceptable and what happens when evidence is uncertain.
1Learn the idea
Read
Worked release rule
The team agrees before testing:
Release when:
- at least 18/20 cases meet the rubric;
- all 4 safety cases pass;
- all 3 unsupported-local-fact cases abstain;
- the science owner approves any changed gold label.
Stop and roll back when:
- a released answer gives actionable unsafe laboratory advice; or
- weekly reviewed accuracy falls below 90% twice.
Decision owner: science product lead.
Safety veto: safeguarding lead.
If a candidate scores 19/20 but fails one safety case, it does not ship. The threshold was chosen to prevent the aggregate score from bargaining away a critical requirement.
Read
Guided practice
The teacher and product owner disagree about an answer on viruses. Write who decides (the content owner), what evidence they inspect (approved curriculum and scientific framing), and what the system does meanwhile (hold the release or mark the case unresolved). “Let the model judge” is not a resolution.
Go deeper
Before you start
Why this matters
For the school science FAQ:
- a science teacher approves factual labels and resolves disputed content;
- a safeguarding lead defines non-negotiable safety cases;
- the product owner decides whether the tool is useful enough to release;
- the delivery team runs the set, preserves results, and rolls back;
- the model may help grade low-risk cases, but it does not approve itself.
Ownership means a person has authority, evidence, and an action—not merely their name in a document.
Related lessons
Check your understanding
Page assessment
Answer from memory. Completion is saved from this evidence, not from opening the next page.
All responses are required.