Trace stakes and incentives: barcode scanner

A misread barcode is visible and usually reversible at the desk. A poor recommendation can quietly make some authors less visible. Both errors matter, but they differ in detectability, scale, and remedy.

For any system, ask:

• Who defines success?
• Who receives the benefit?
• Who experiences errors or exclusion?
• Can affected people understand and challenge an output?
• Who can pause the system?

The recommendation supplier reports increased clicks. Mira asks whether clicks represent the library’s goals. A list designed only for clicks may repeatedly show already popular titles. Staff sample recommendations across age ranges, languages, and catalogue sections. They find that local-language books rarely appear.

Mira does not claim the system is intentionally unfair. She records a measurable mismatch: the optimization signal and the library’s collection-access goal differ. The team adds catalogue-coverage measures, a “show me something different” option, and a staff owner for complaints.

“The algorithm is neutral because it is mathematical.” Mathematics implements choices: what to predict, which data to use, which errors matter, and when an output is acted on.

For the barcode scanner, answer the five questions above. Then do the same for recommendations. Which system needs broader monitoring, and why?

Take an automated decision from school, work, or a public service. Draw a consequence chain from output to action to affected person. Mark where correction is possible. If no correction point exists, propose one.