Mastery: build an automation blueprint

A team wants to automate hiring. Which is the strongest first proposal?

A. Let AI decide which applicants deserve interviews.
B. Let AI run the entire hiring process because it is faster.
C. Extract scheduling constraints from recruiter-approved emails into a draft calendar form.
D. Ask AI to invent a hiring policy.

Answer: C. It is a bounded, reversible administrative slice with an observable output and a natural review point. Applicant selection has material fairness, legal, and human consequences. A broad goal such as “automate hiring” hides distinct tasks and risks.

The task card should still define allowed inputs, privacy, required fields, uncertainty behavior, and deletion rules.

An invoice workflow must identify the supplier, categorize the purchase, calculate total tax, verify that the amount is below an approval threshold, select the approver, and schedule payment. Which steps are reasonable AI candidates?

Answer: AI may help identify the supplier from variable layouts and categorize a free-text purchase description. Deterministic code should calculate tax from authoritative numeric fields, compare thresholds, select approvers from policy, and control payment scheduling. Validation and a human gate should protect consequential exceptions.

The principle is not “AI first, rules second.” It is to place interpretation inside a deterministic envelope.

An AI drafts a refund message, the workflow issues the refund, and an employee approves the message before it is sent. What should change?

Answer: Move approval before the refund, and show the employee the exact amount, destination, source transaction, policy basis, and proposed communication. Bind approval to those versions. Recheck relevant state at execution. If only the message requires review while the refund is already authorized by a deterministic rule, describe those controls separately.

Review after the side effect is auditing, not pre-action approval.

A record-creation API times out. Is retrying the entire workflow a safe response?

Answer: Not until the system knows whether the record was created. Reconcile using an idempotency key or query. If the first call succeeded, retrying can duplicate the record. Preserve step state and retry only the unresolved action. If outcome remains unknown, pause or escalate rather than claiming failure or success.

A classifier reports 96% overall accuracy. What else is required before automated routing?

Answer: At minimum: representative data, per-category precision and recall, error severity, abstention and coverage, subgroup analysis, operational fallback, end-to-end latency and reliability, cost per completed case, comparison with baseline, monitoring, and rollback thresholds. A dominant easy category can make overall accuracy look strong while a rare critical category fails.

Complete every section below.

1. Outcome and baseline

State the user or operational outcome without mentioning AI. Name current volume, handling and elapsed time, quality, error and rework, backlog, cost, and relevant experience measures. Identify the smallest useful automation slice and explicitly list what remains out of scope.

2. Trigger and terminal states

Name the observable event, unique identifier, duplicate behavior, and eligibility rules. Define every terminal state: completed, no action needed, rejected, cancelled, expired, failed, quarantined, or escalated. No case should disappear into a generic processed state.

3. Data and trust boundaries

List each input, source of truth, freshness requirement, sensitivity, permitted use, and retention. Separate untrusted text from system instructions. Specify data minimization and access. State what the workflow must never infer when an authoritative lookup is required.

4. Step map

For every node, record input, operation, output, next state, owner, evidence, timeout, and failure branch. Label the node deterministic, AI, human, or external system. Explain why AI is useful for each AI node and why an ordinary rule is insufficient.

5. AI contracts and deterministic invariants

Define allowed output schemas, categories, evidence requirements, uncertainty routes, model and prompt versioning, and evaluation cases. Then list hard invariants enforced in code: thresholds, allowlists, arithmetic relationships, authorization, valid state transitions, idempotency, and side-effect limits.

6. Human gates

Name the exact side effect that waits. Identify reviewer role, evidence, available choices, timeout, escalation, workload estimate, and proposal version. Explain what invalidates approval. Ensure permissions prevent execution without a valid approval record.

7. Failure and recovery

Cover malformed input, valid-but-wrong interpretation, duplicate trigger, timeout after possible success, stale state, unavailable dependency, exhausted quota, unsafe content, queue overload, and operator absence. For each, define detection, containment, recovery, owner, and user-visible state.

8. Evaluation and rollout

Choose task-quality, workflow-reliability, risk, experience, and economic metrics. Define the baseline, test sets, shadow mode, suggestion phase, narrow production slice, and release thresholds. Write pause, rollback, and expansion criteria before launch.

9. Operations and accountability

Specify trace IDs, logs, dashboards, alerts, retention, access, on-call ownership, kill switch, manual fallback capacity, incident response, and change management. Explain how overrides, incidents, and sampled reviews update evaluations without treating every production action as unquestioned ground truth.

A fixed workflow is usually the right design when the sequence and branches are known. An agent becomes relevant when observations determine a path that cannot be economically enumerated in advance—for example, investigating an unfamiliar technical failure by choosing among bounded diagnostic tools.

Do not relabel a workflow as an agent because it contains AI. The inbox example follows authored steps and routing rules. If a future version can decide whether to search an account, ask a clarifying question, inspect documentation, or stop based on evolving state, it introduces agentic control. That flexibility also introduces variable cost, latency, action order, and failure paths. It needs tool allowlists, step budgets, stopping criteria, traces, and success checks.

The blueprint remains useful: deterministic rules should still control permissions and side effects. Human gates should still protect consequential actions. Prefer the least variable architecture that meets the requirement.

Human review is not a temporary patch that automatically disappears as model quality rises. Some gates exist because accountability, values, regulation, or consequence requires a person. Others can evolve through staged autonomy when representative evidence shows stable quality and recovery works.

For the next design layer, classify actions by impact, scale, reversibility, detection speed, and uncertainty. Choose among approval, edit, escalation, sampled review, and dual control. Then test the reviewer experience and queue capacity as seriously as model performance.