Evaluate with evidence

Track for model selection: field-level F1, schema-valid rate, $/1k docs, p95 latency, truncation rate, fallback success. Report fractions like 36/40, not vague quality adjectives. Segment by language, plan tier, document length, or other slices that matter for the invoice-extraction service.

Freeze inputs and neighboring versions while evaluating model selection. Change one control. Pair results case by case on the invoice-extraction service. Define hard gates (severe errors, privacy, latency) before the bake-off. Use deterministic checks where possible; humans for nuance; model judges only with calibration against gold.

Numeric reminder for model selection: If model L hits 96% field-F1 at $0.12/1k docs and model H hits 97% at $0.55/1k, the +1 pt may lose unless errors are extremely costly.

Resist adding a twelfth metric before the first three for model selection on the invoice-extraction service have owners. This workload improves faster when a small scorecard is trusted than when a warehouse of unused plots exists.

Also pin one numeric memory from this model selection chapter: If model L hits 96% field-F1 at $0.12/1k docs and model H hits 97% at $0.55/1k, the +1 pt may lose unless errors are extremely costly. That number is not decoration; it is a template for how claims about model selection on the invoice-extraction service should look in design docs. Scoped specifically to model selection / invoice-extraction service / evaluation.

People confuse model selection with neighboring buzzwords when debugging the invoice-extraction service. Before changing prompts, ask whether the broken stage was evidence gathering, the model selection judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried model selection and it failed”) that blocks the next team on the invoice-extraction service. Scoped specifically to model selection / invoice-extraction service / evaluation.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to choosing a model rather than generic AI advice.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to choosing a model rather than generic AI advice.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to choosing a model rather than generic AI advice.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to choosing a model rather than generic AI advice.