Page 6 of 8~104 min topic

Prediction: your first ML idea

Print so you can see what happened

Observability for a toy game means leave a trail a teammate can read — not secret glow logs nobody checks.

~13 min this pageTesting and observability

1Learn the idea

Read

Make the scoreboard speak

The print line is your event log for this lab. Structured text beats “it worked :)” because you can compare it to the expected evidence character by character.

truth = [1, 0, 1, 1, 0]
scores = [0.8, 0.3, 0.6, 0.9, 0.2]
threshold = 0.5
preds = [int(score >= threshold) for score in scores]

pairs = list(zip(truth, preds))
tp = pairs.count((1, 1)); fp = pairs.count((0, 1))
tn = pairs.count((0, 0)); fn = pairs.count((1, 0))
accuracy = (tp + tn) / len(pairs)
print(preds, f"accuracy={accuracy:.2f}", tp, fp, tn, fn)

Expected evidence:

[1, 0, 1, 1, 0] accuracy=1.00 3 0 2 0

Read

Signals a classmate can debug from

Design a tiny checklist beside the print:

  • threshold — was it still 0.5?
  • preds — do guesses match hand counting?
  • tp fp tn fn — do they sum to 5?
  • accuracy — does it match (tp+tn)/5?
  • label the run fixture-only so nobody mistakes it for a huge real dataset.

Turn one past bug (mismatched lengths, threshold out of range) into a test that fails if the warning disappears. Observability without a failing check is decoration; with a check, it is part of the toy.

Read

Privacy in the trail

Do not print real inbox text, phone numbers, or passwords. Print scores, truths, and counts. If you demo “spam message?” use made-up strings like "FREE PRIZE!!!" — never a friend’s real chat.

Read

Make three stories distinguishable

Imagine a teammate opens only your printouts after something went weird. They should be able to tell apart:

  1. Wrong lists — lengths mismatched or empty, so the scoreboard never should have run.
  2. Wrong cutoff — threshold drifted, so guesses flipped even though truths stayed put.
  3. Honest fixture win — the expected evidence line appears, labeled practice-only.

If those three stories look identical in your notes, rename fields until they do not. Good observability is less about more printing and more about printing the few facts that settle arguments. Add a one-line fixture_id (for example five-round-spam-v1) whenever you change a truth or score so old screenshots do not confuse new runs.

Go deeper

Before you start

Why this matters

Your friend runs the spam-or-shot mini game after you leave club. They only see the final screen. Write three fields that must appear so they can tell: (1) which cutoff was used, (2) what the guesses were, (3) whether the four buckets and accuracy match the five-round fixture. What field must never appear (a real classmate’s message text)?

Check your understanding

Page assessment

Answer from memory. Completion is saved from this evidence, not from opening the next page.

1. Can someone debug from the printout alone?
2. Are sensitive classmate messages absent from logs?
3. Is the run labeled as a tiny fixture, not a big real test?
4. Did you connect at least one historical bug to a check?

All responses are required.