Chapter CHuman in the loopPage 3 of 8

Human in the loop

Where to place the gate

A review step protects only the actions that occur after it, so gate placement matters as much as having a gate.

~13 minHow it works

Before you start

Why this matters

Suppose an AI reads a customer request, retrieves account data, drafts a refund decision, updates the billing system, and sends an email. A reviewer who checks the email after the database update cannot prevent an incorrect refund. A reviewer who approves the refund but never sees the customer’s evidence may approve a well-written mistake. Draw the workflow before deciding where “human review” belongs.

1Learn the idea

Read

Trace the full action chain

Write the process as verbs connected by arrows:

Receive → classify → retrieve → reason → propose → approve → execute → notify → monitor

Real systems may have more steps, but this chain exposes an important fact: an AI output often triggers several downstream actions. Each boundary can be a control point.

For every step, record:

  • what enters the step;
  • what changes;
  • whether an external side effect occurs;
  • whether the change is reversible;
  • what evidence is available;
  • who owns the decision.

“Generate response” is not a complete workflow if generating the response automatically calls a tool. Separate thinking, proposing, and acting. An agent may appear to produce text while it is also sending messages, creating records, or spending money.

Read

Gate before the first irreversible side effect

A strong default is to place review immediately before the first costly or irreversible side effect. Let the AI do cheap preparation—searching, summarizing, drafting, checking rules—then pause before it sends, transfers, publishes, deletes, or changes authoritative data.

This placement gives the reviewer a nearly complete proposal without allowing the proposal to become reality. It also reduces wasted human effort: people review a specific action rather than manually perform every preparation step.

There are exceptions. If retrieval itself exposes restricted data, authorization must happen before retrieval. If a draft includes private information that the reviewer is not allowed to see, access control belongs earlier. Gates cannot repair a permission violation that already happened.

Read

Give reviewers the decision packet

A reviewer needs a compact decision packet, not only the AI’s conclusion. A useful packet includes:

  1. the original request or event;
  2. the proposed action;
  3. the key facts and sources;
  4. policy rules and thresholds that apply;
  5. missing or conflicting information;
  6. the expected effect of approval;
  7. a trace or version identifier.

More information is not always better. Dumping a fifty-page transcript can hide the relevant fact. Summarize, but let reviewers open the original evidence. Clearly distinguish source facts from model-generated interpretation.

Avoid presenting an AI confidence number as the only reason to approve. Model probabilities may not be calibrated for your workflow, and a high score does not lower the impact of a rare error.

Read

Balance latency and safety

Every gate adds waiting time. That matters in live support, fraud detection, and incident response. Do not solve latency by quietly removing review. Design service levels and branches.

For example:

  • low-risk requests proceed automatically and are sampled later;
  • medium-risk requests wait up to ten minutes for review;
  • high-risk requests remain paused until a specialist responds;
  • urgent safety events follow a preapproved emergency policy.

Measure queue time, review time, and decision quality separately. A slow queue may require staffing or better routing. Long review time may mean the decision packet is poor. High override rates may indicate that the automation is proposing the wrong action.

Timeout behavior must be explicit. “If no one answers, send anyway” is automation with a temporary delay, not approval. For consequential actions, safer defaults are often pause, cancel, or escalate.

Read

Avoid review overload

When every event is flagged, attention becomes the scarce resource. Reviewers may skim, approve by habit, or miss the truly dangerous case.

Reduce overload by:

  • separating routine cases from exceptions;
  • grouping related low-risk items into batches;
  • ranking cases by consequence, not only model uncertainty;
  • removing duplicate alerts;
  • setting workload limits and backup ownership;
  • improving upstream data so fewer cases are ambiguous.

Monitor approval speed and override patterns. An abrupt rise in one-click approvals can signal fatigue. A growing queue can turn a safe design into an unavailable service. Human capacity is part of system capacity.

Read

Add a gate after action too

Pre-action review prevents mistakes. Post-action monitoring detects failures that passed review or happened in automated branches. Good systems often use both.

After execution, record the result, watch for complaints or anomalies, and make rollback possible. Review samples of approved actions to check reviewer consistency. A human decision is not automatically correct; it is an accountable judgment that can also improve through feedback.

Read

Draw your gate map

Use three marks on a workflow:

  • H for a required human decision;
  • A for an automated policy check;
  • M for monitoring after action.

Then ask whether any side effect occurs before the relevant H or A. Ask what happens if the reviewer is unavailable. Ask whether monitoring can identify and correct the failure quickly enough.

Checking tutor…

Continue learning · glossary & guides