Instrument the tiny linear model

Instrument the hours→score linear fit with holdout so a run records enough structure to debug offline: counts, latency if relevant, pass/fail of holdout absolute error, and a stable stage name. Redact secrets and raw credentials from every event.

import json
print(json.dumps({'slope':1.4,'mae':0.2,'baseline_mae':1.5,'holdout':True}))

Expected evidence: fit metrics vs baseline. Prefer JSON or structured text you can grep in CI over prose logs for train-a-tiny-model.

Turn one historical failure—especially training on the test row—into a test that fails if the signal disappears for the tiny linear model. Observability without a failing test is optional decoration; observability with a test is part of the train-a-tiny-model artifact.

Draft a three-field event for the tiny linear model: stage, ok, and one domain field derived from holdout absolute error; slope sign matches the hours/score trend. Add fixture_id or docs_version when content can change. Explicitly list fields that must never appear (tokens, passwords, raw prompts) because publishing per-learner residuals that re-identify students is in scope for this lab.

Wire one assertion that fails if the tiny linear model event is missing after a run. Observability that cannot fail a test will not survive contact with a busy train-a-tiny-model repository.

Imagine a teammate opens only your event stream after a bad deploy. Could they tell whether small hours/score table with one held-out pair was wrong, whether training on the test row, or reporting R² without the holdout error returned, or whether publishing per-learner residuals that re-identify students slipped through? If not, rename fields until those three stories are distinguishable.

At the testing and observability stage for train-a-tiny-model, the job is narrower than finishing a product demo. You are creating one progressive evidence piece about small hours/score table with one held-out pair that later pages inherit without redefining success. Keep that fixture small enough to inspect by hand, keep outputs copy-pasteable as text, and refuse to narrate this baseline as if it were a production SLA: predict mean train score for the holdout before fitting.

For this page specifically, success looks like a structured event schema locked by a test while still centering the user decision to estimate slope/intercept from study hours and score a held-out row honestly. If you cannot point to a file, command, or assertion that proves that for the tiny linear model, stay on this page instead of advancing.

Overfitting glossary

Previous · Next