Measure whether the score labeler works

Translate the claim into assertions or a tiny eval harness. The metric to protect is: boundary labels at 0.49/0.50/0.51 and empty-list positive count 0. Always record the denominator (how many cases) beside any rate. A percentage without a denominator is marketing, not measurement.

def label_scores(scores, threshold):
    labels = ["yes" if score >= threshold else "no" for score in scores]
    return labels, labels.count("yes")

assert label_scores([0.49, 0.50, 0.51], 0.50) == (
    ["no", "yes", "yes"],
    2,
)
assert label_scores([], 0.50) == ([], 0)
assert label_scores([0.0, 1.0], 0.50) == (["no", "yes"], 1)
print("3/3 checks passed")

Expected output: 3/3 checks passed.

To prove the boundary test can detect a real bug, temporarily change >= to >. The first assertion must fail because 0.50 becomes "no". Restore >= and rerun. A test that stays green under that mutation does not protect the agreed equality rule.

Be explicit: beating the baseline (hand-label the three scores on paper before writing the function) on this fixture does not prove behavior under string scores that compare lexicographically, or IndentationError that hides a wrong cutoff. Label observations separately from conclusions so the next page inherits honest evidence about the score labeler.

Compute boundary labels at 0.49/0.50/0.51 and empty-list positive count 0 with the denominator written beside the rate every time. For this chapter, the evaluation set is intentionally tiny; that is allowed only if you say so in the evidence. Compare against hand-label the three scores on paper before writing the function before celebrating.

Add one negative case aimed at string scores that compare lexicographically, or IndentationError that hides a wrong cutoff. A suite with only happy cases cannot protect the score labeler when the characteristic failure appears in review.

If a check is expensive or flaky, shrink it until it is deterministic on scores=[0.2,0.9,0.4], threshold=0.5. Flaky green builds teach the team to ignore gates. Record what this page does not prove so security-ops and mastery-ship inherit honest limits.

Test a shipping-fee function whose free-shipping boundary is exactly $50. Include $49.99, $50.00, $50.01, and an empty cart. Mutate the boundary operator and confirm the suite catches it.

ML Python starter

Previous · Next