Mastery check and ship criteria

An AI drafts three subject lines for an unsent internal newsletter. Which starting control is most proportionate?

A. Dual control from two executives
B. Draft-only use with ordinary editing
C. Automatic publication to all customers
D. Permanent ban on AI

Answer: B. The output is reversible and has no external side effect until someone uses it. Ordinary editing is enough. The control should become stronger at the publication step, especially if the audience or content creates reputational risk.

The lesson is to classify the exact action. “AI writes marketing” is too broad. Drafting, approving claims, selecting recipients, and sending are different actions.

An AI reviews a billing complaint, updates the account, and then generates a message. The team currently reviews the message before sending it. What is wrong?

Answer: The account update occurs before the review gate. The reviewer can prevent a bad message but cannot prevent the incorrect account change. Move the gate before the first consequential side effect, or separate approval of the account action from editing of the message.

Also confirm that the reviewer sees source evidence and the exact proposed update. Reviewing polished prose alone does not validate the underlying action.

A document classifier processes 100,000 internal, non-sensitive files per day. Labels are reversible, quality is measurable, and one wrong label has low impact. Which pattern is a reasonable starting candidate after validation?

Answer: Automated labeling with random and risk-based batch review can be proportionate. Required approval of every label would create enormous cost without matching the consequence.

The team still needs rollback, monitoring, representative evaluation, and a safe branch for unusual or sensitive files. Low impact does not mean no controls.

A high-value transfer waits for human approval. If nobody responds within fifteen minutes, the system approves automatically to meet a speed target. Is this human approval?

Answer: No. It is delayed automation. The human can intervene, but approval is not required. If the transfer truly requires human authorization, timeout should pause, cancel, or escalate according to policy. If delayed automation is acceptable, describe it honestly and control it as an automated action.

Timeout behavior often reveals the real system design. Write it before launch, not during an incident.

A support assistant achieved 98% accuracy in one month of reviewed cases. Name at least four questions to answer before removing review.

Answer: Strong answers include:

• Was the test set representative of production and important subgroups?
• What did the two-percent error contain, and how harmful was it?
• Are results stable by request type, language, customer, and time?
• Can deterministic policy checks contain unsafe actions?
• Are monitoring, alerts, limits, and rollback tested?
• What changes require reevaluation?
• Can the human fallback absorb volume after a pause?
• Is the proposed automated slice narrow and reversible?

An average accuracy number is not a release decision.

Do not ship a consequential AI workflow until the team can answer these items.

Action and risk

• The exact side effects are named as verbs.
• Impact, scale, reversibility, detection speed, and uncertainty are assessed.
• High-risk and legally or professionally constrained decisions have expert review.
• Permissions prevent the model from taking actions outside its role.

Human decision

• The review pattern is explicit: approve, edit, escalate, batch review, or dual control.
• The reviewer’s decision and responsibility fit in one clear sentence.
• Reviewers have the expertise, authority, time, and information required.
• Available choices include safe rejection and escalation paths.
• Timeout behavior is documented and tested.

Evidence and interface

• The interface separates source facts from generated interpretation.
• Reviewers see the proposed side effect before it occurs.
• Missing, conflicting, or unusual information is prominent.
• The interface does not use confidence or polished wording to pressure approval.
• Accessibility and workload have been tested with actual reviewers.

Execution and recovery

• Deterministic rules enforce hard limits where possible.
• Execution success is verified before a success message is sent.
• Retries are safe, including idempotency where needed.
• Rollback or remediation is possible and has an owner.
• The team can pause or narrow automation without emergency code changes.

Accountability and learning

• Proposal, evidence versions, review, execution, and outcome share a trace ID.
• Sensitive audit data has access and retention controls.
• Reviewer edits and overrides have meaningful reason categories.
• Quality is measured by risk group, not only overall average.
• Incidents and sampled reviews feed evaluations and process improvements.

Reject “a human will monitor it” unless the design names alerts, response time, authority, and safe defaults. Reject “high confidence goes automatic” unless confidence is calibrated and combined with action risk. Reject approval interfaces that hide evidence or make rejection difficult. Reject audit trails that store only the final output. Reject autonomy plans with no rollback trigger.

Most importantly, reject the idea that adding a person at the end automatically makes a system responsible. Safety comes from the whole control loop: bounded permissions, good evidence, meaningful decisions, reliable execution, monitoring, and recovery.

In code, these ideas become tool allowlists, policy checks, typed action proposals, approval state machines, idempotent execution, append-only events, role-based access, and monitoring. The next practical step is to build a workflow in which the model can propose an action but cannot execute it until an independent approval record exists.

Guardrails and human review complement each other. Code handles exact, repeatable limits at machine speed. People handle context, ambiguity, values, and exceptions. Neither should silently substitute for the other.