Mastery: connect the pieces

1. Idea — Training updates parameters from examples; inference freezes them and predicts. Prompts and retrieval change temporary context, not weights.…
2. Mechanism — During training, compute loss on batches and apply optimizer steps across epochs. At inference, run forward passes only, often with batching and quantization fo…
3. Controls — learning rate, batch size, epochs, regularization, checkpoint pick, inference precision, decode settings
4. Tradeoff — Training is expensive but amortized across many uses and can change persistent behavior. Inference is repeated per request and dominates operating cost at scale…
5. Failures — Confusing prompt edits with training; Shipping an overfit checkpoint
6. Metrics — train/val metrics, steps, inference latency, $/1k inferences, refresh lag for facts
7. Ship rule — Freeze best-val checkpoint after 5 epochs; serve quantized forward pass; put policy text in prompts/RAG not weights.

Neighboring layers (retrieval, serving, policy, human review) still own what training vs inference cannot on the ticket classifier service. Do not ask this chapter’s dial to replace permissions or source truth.

Mastery is transfer: take training vs inference into a second scenario that is not the ticket classifier service and rebuild the checklist without copying sentences. If you can only recite this chapter’s examples for training vs inference, you have memorized a story, not a model.

Also pin one numeric memory from this training vs inference chapter: one epoch over 80,000 examples with batch size 100 requires 800 optimizer steps; five epochs require 4,000 steps That number is not decoration; it is a template for how claims about training vs inference on the ticket classifier service should look in design docs. Scoped specifically to training vs inference / ticket classifier service / mastery-connect.

People confuse training vs inference with neighboring buzzwords when debugging the ticket classifier service. Before changing prompts, ask whether the broken stage was evidence gathering, the training vs inference judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried training vs inference and it failed”) that blocks the next team on the ticket classifier service. Scoped specifically to training vs inference / ticket classifier service / mastery-connect.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to training vs inference rather than generic AI advice.