Tuning only on answers you already saw

1. Tuning on the practice answers you already saw — every knob twist is judged on the same rows, so you memorize the evaluation itself.
2. Peeking at the final test while “just experimenting” — the held-out set stops being held out.
3. Tiny, samey data — five nearly identical spam texts; the model learns your typing quirks, not spam.
4. Celebrating only training accuracy — hides the surprise-quiz crash.
5. Changing five knobs at once — you cannot tell which change helped or hurt.

These failures look like progress because practice numbers rise. The damage shows up later, when a friend tries different examples.

Start: 99% train / 84% val. You keep tuning only against the training sheet until train hits 100% and val slides to 80%. You “improved” the wrong scoreboard. A healthier path uses validation (or a friend’s fresh examples) to decide when to stop, aiming toward something more like 93/91.

• Freeze a validation pack before heavy tuning.
• Touch the final test once at the end (or keep it with the teacher).
• Log each change: what knob, what train score, what val score.
• If val does not improve after several tries, stop and simplify instead of grinding practice perfection.

Near-miss: you tune on validation a little — risky but named. Counterexample: you tune on the final test nightly and still call it “unseen.” Label which workflow you actually used so your write-up stays honest.

Tuning on answers you already saw feels productive: every evening the graph goes up. Friends may cheer. Teachers may see a polished demo. The failure mode thrives on that applause. Build a habit that feels slightly worse in the moment: keep a sealed pack, score it less often, and treat a flat validation curve as useful news.

Add a notebook rule: “If I changed a knob, I must write which set I measured on.” Without that sentence, future-you will forget and retell a heroic training curve that was never a fair test. When a friend’s different examples suddenly score much lower, do not blame the friend first — ask whether you polished only the practice answers you already saw.