Practice: design three gates

Copy these prompts:

1. Action: What exact verb creates the side effect?
2. Risk: What are impact, scale, reversibility, detection speed, and uncertainty?
3. Pattern: Approve, edit, escalate, batch review, or dual control?
4. Gate: Immediately before which step?
5. Packet: What must the reviewer see?
6. Options: What can the reviewer decide?
7. Timeout: What happens when nobody responds?
8. Record: What evidence and versions are logged?
9. Monitor: Which outcomes trigger investigation or rollback?

The goal is not maximum review. The goal is a control proportionate to the consequence.

An AI transcribes a product meeting, summarizes decisions, and posts the summary to an internal project channel. The meeting sometimes includes customer names, unreleased plans, and informal ideas that are not decisions.

Design the workflow.

Consider whether transcription, summarization, and posting have the same risk. Decide whether every summary needs editing or whether lower-risk meetings can be sampled. Explain how the system distinguishes a decision from discussion.

Solution rubric

A strong answer separates draft from publish. The AI may transcribe and draft inside an access-controlled workspace, but a participant edits or approves before posting. The reviewer sees the recording or transcript around each claimed decision, attendees, proposed owners, and detected sensitive terms.

The reviewer can edit, remove sensitive content, mark an item as discussion rather than decision, request clarification, or approve. Timeout means the summary remains a private draft; it is not posted automatically.

The log records the source meeting, transcript version, generated draft, edits, reviewer, destination channel, and publication result. Monitoring samples approved summaries for incorrect decisions and checks whether sensitive information reaches unauthorized channels.

A mature low-risk branch might auto-post summaries for predefined meetings that never contain restricted material, with sampling and an easy correction path. That expansion requires evidence, not an assumption that “internal” means harmless.

A consumer health app asks users about symptoms and generates guidance. It can suggest self-care information, recommend contacting a clinician, or tell the user to seek urgent help. It cannot diagnose.

Design the human role without making a clinician approve every ordinary response in real time.

Consider that delay can itself create harm. Identify which parts should be deterministic, which should be reviewed before launch, and which live cases need escalation.

Solution rubric

A strong answer does not route every user through a slow approval queue. Emergency warning signs use validated deterministic rules and approved language that immediately directs the user to local emergency services. The generative model must not weaken or override those rules.

The AI can provide bounded educational content for low-risk inputs using reviewed sources. Ambiguous, conflicting, high-risk, or unsupported cases escalate to a qualified service if one is actually available; otherwise the product clearly states its limits and directs the user to appropriate care.

Clinical and safety experts review content, rules, and representative scenarios before deployment. Live human review is reserved for a service that is staffed, trained, and able to respond within a defined time. Timeout must never imply that silence equals safety.

The decision packet includes user-provided symptoms, triggered rules, source content, missing information, and the proposed guidance. Logs minimize sensitive data, restrict access, and preserve the rule and content versions. Monitoring watches for missed urgent cases, unsafe reassurance, subgroup performance, complaints, and changes in input patterns.

An answer that says “show model confidence and let the user decide” is weak. Users cannot evaluate hidden model uncertainty, especially under stress.

An online marketplace uses AI to detect suspicious seller activity. The system can hold a weekly payout for investigation. A false hold may prevent a small business from paying staff; a missed fraud case may lose money.

Design a gate for holding payouts and a path toward bounded automation.

Solution rubric

A strong answer separates risk scoring from the hold action. The AI gathers signals and recommends a hold. Deterministic policy checks verify account identity, amount, prior cases, and allowed reasons. A trained fraud reviewer approves high-value or ambiguous holds before the payout deadline.

The packet shows the payout, triggered signals, evidence history, known false-positive risks, policy, and the exact effect and duration of the hold. The reviewer can approve a temporary hold, release, request evidence, or escalate. High-value holds may require dual control.

Timeout behavior depends on the documented policy and legal context; it should not be invented by the model. If the organization cannot review in time, a bounded short hold with prompt notice and appeal may be safer than an indefinite block, but the policy must be approved by responsible experts.

The audit trail records signal and policy versions, evidence, reviewer decisions, notices, appeals, release, and final outcomes. Monitor false holds, fraud loss, appeal reversals, time to resolution, and performance across seller groups.

Staged autonomy might begin with very low-value payouts matching a narrow, well-tested fraud pattern. Use daily limits, sampled review, canary traffic, and immediate rollback after harmful false positives.

The meeting scenario favors edit-before-publish because mistakes are reversible but privacy still matters. The health scenario uses strong predeployment human expertise and deterministic urgent rules because waiting for live approval can be dangerous. The payout scenario requires case-level approval or tightly bounded automation because the system changes access to money.

HITL is not one component copied everywhere. The human role changes with the action, timing, expertise, and available evidence.