Learn the controls and knobs

Primary knobs for model selection: quality threshold, p95 latency budget, cost ceiling, context length need, structured-output support, data residency, fallback model.

Write a sheet for the invoice-extraction service with columns: control, current value, predicted benefit, predicted cost, rollback trigger. Fill it using this topic’s real tension: Larger models often raise quality and cost/latency together. Tiny models need more scaffolding. Switching vendors can win price and lose schema features.

Change one model selection family at a time. If you move two knobs and the invoice-extraction service improves, you learned a cocktail, not a cause—and you cannot roll back surgically.

End users of the invoice-extraction service should see only safe dials related to model selection. Infrastructure limits, private prompts, and policy thresholds stay server-owned. A user-facing control that bypasses those limits is a vulnerability dressed as UX for model selection.

Publish the model selection control sheet next to the invoice-extraction service runbook. On-call should see which knob moved in the last deploy without reading chat archaeology. Unknown model selection knobs are unowned knobs.

Also pin one numeric memory from this model selection chapter: If model L hits 96% field-F1 at $0.12/1k docs and model H hits 97% at $0.55/1k, the +1 pt may lose unless errors are extremely costly. That number is not decoration; it is a template for how claims about model selection on the invoice-extraction service should look in design docs. Scoped specifically to model selection / invoice-extraction service / controls-and-knobs.

People confuse model selection with neighboring buzzwords when debugging the invoice-extraction service. Before changing prompts, ask whether the broken stage was evidence gathering, the model selection judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried model selection and it failed”) that blocks the next team on the invoice-extraction service. Scoped specifically to model selection / invoice-extraction service / controls-and-knobs.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to choosing a model rather than generic AI advice.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to choosing a model rather than generic AI advice.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to choosing a model rather than generic AI advice.