Mastery: connect the pieces

A complete explanation of Structured outputs has four parts. First: Structured output constrains a model response to a machine-readable contract such as JSON Schema. It solves syntax and shape problems; it does not guarantee that the values are true, safe, authorized, or semantically consistent. Second, the mechanism: The application defines allowed fields, types, enums, required properties, and whether extra properties are forbidden. Some providers constrain token decoding to valid schema paths; others offer JSON mode that guarantees only parseable JSON. The application still parses, validates, applies business rules, and handles refusal or truncation. Third, the operational controls: schema strictness; required versus optional fields; enums; numeric bounds; nullable fields; additionalProperties; schema version; retry policy; validation errors; and separation between extraction and action. Fourth, the evidence: Measure parse success, schema-valid rate, field-level precision and recall, enum validity, business-rule pass rate, refusal rate, repair attempts, latency, and downstream execution errors. Test missing, ambiguous, adversarial, and oversized inputs.

Use the scenario as an oral exam: An invoice extractor returns {invoice_id, currency, total_cents, due_date, confidence}. The schema requires integer cents and ISO dates; a business rule checks that line-item cents sum to total_cents before any payment workflow sees it. Defend one design choice, then argue against it using this tradeoff: Strict schemas simplify downstream code but can make partial answers harder. Large nested schemas consume tokens and reduce model reliability. Retries can repair transient failures but may duplicate side effects unless generation and execution are separate. Finally, identify which of these failures your design catches and which remain: confusing JSON mode with schema adherence, accepting a valid but invented account ID, silently coercing types, allowing unknown fields, changing a schema without versioning, executing before validation, recursive schemas unsupported by the provider, and retrying a tool action rather than only regeneration.

Mastery is not recalling every term. It is predicting consequences before running the system, noticing when evidence contradicts the prediction, and revising the design without moving the goalposts. Keep a decision record containing the workload, baseline, configuration, test set version, results, known limitations, owner, and rollback condition.

An invoice extractor returns {invoice_id, currency, total_cents, due_date, confidence}. The schema requires integer cents and ISO dates; a business rule checks that line-item cents sum to total_cents before any payment workflow sees it.

The worked number is if 970 of 1,000 outputs parse and 930 pass the schema, parse rate = 97% but end-to-end schema-valid rate = 93%; report both. State the unit and denominator whenever you report it. A percentage without a denominator can conceal a tiny sample; a latency without a percentile can conceal slow users; a similarity score without a labeled task can conceal irrelevant neighbors. Compare the observed value with a threshold chosen before seeing the final test result.

Now test the tempting shortcut. Suppose the team optimizes only the most visible metric. The result may look better while the system becomes less trustworthy. The reason is concrete: Strict schemas simplify downstream code but can make partial answers harder. Large nested schemas consume tokens and reduce model reliability. Retries can repair transient failures but may duplicate side effects unless generation and execution are separate. This is why the decision record must include both the intended gain and the tolerated regression. If the tolerated regression is unknown, the change is not ready for a consequential workflow.

• Prefer a measured baseline over a persuasive demo.
• Keep versions, inputs, and thresholds reproducible.
• Separate syntactic success from semantic correctness and authorization.
• Escalate or abstain when evidence falls outside the contract.
• Re-evaluate when data, traffic, models, providers, or user goals change.

These rules turn the topic into an engineering decision rather than a slogan. They also make disagreement productive: another person can challenge the assumptions, rerun the evaluation, and reach a documented conclusion.

Give a three-minute teach-back with no slides. Minute one: define the technique and its boundary. Minute two: trace the mechanism using the worked case and calculation. Minute three: defend the chosen controls with evaluation evidence, then name the strongest unresolved failure. Ask the listener to change one assumption and update your recommendation aloud. You have mastered the topic when the recommendation changes for a technical reason—not because the vocabulary changed—and when you can specify the next experiment that would reduce the most consequential uncertainty.