Chapter AAI and human capabilityPage 3 of 8

AI and human capability

Recognise real-world forms: sorting warehouse parcels

AI and human capability becomes understandable when you compare performance on a defined task instead of ranking general intelligence; on this page, the example of sorting warehouse parcels makes that boundary concrete.

~13 minRecognise real-world forms — variation across settings

Before you start

Why this matters

Imagine sorting warehouse parcels appearing in an ordinary day. Write down what enters the system, what operation is performed, what comes out, and who acts next. Do not use “the AI knows” as an explanation. For this stage, concentrate on variation across settings. Circle the first detail you would need to observe rather than assume.

Now alter one condition in the scene: the user has an uncommon need, the environment is noisy, the deadline is shorter, or the result affects access to something important. Predict which part of the path changes. This comparison prevents a product label from standing in for evidence about a particular use. This sorting warehouse parcels example is being used here to test variation across settings.

1Learn the idea

Read

The page's central lens

See it

Different strengths

Humans excel

Values · accountability · lived context · taste

AI excels

Speed · scale · pattern match · draft volume

AI is fast at patterns · humans own meaning, stakes, and care

The durable idea is to compare performance on a defined task instead of ranking general intelligence. Applied to sorting warehouse parcels, that means naming a bounded purpose before praising or rejecting the technology. The same technique can be impressive in one setting and unacceptable in another because consequences, available fallbacks, and opportunities for correction differ. Capability is therefore a relationship among a system, a task, a population, and conditions.

Consider the course case: A clinic assigns transcription to software but keeps diagnosis with clinicians. The team should not ask only whether the output looks convincing. It should collect task tests and accountable review, identify who bears an error, and decide who has authority to pause the use. The key limitation is that speed on patterns is not judgment about goals or values. That limitation is not a reason for panic; it is a reason to match confidence and oversight to evidence. This sorting warehouse parcels example is being used here to test variation across settings.

Read

A contrasting example

Compare sorting warehouse parcels with negotiating a family disagreement. The first emphasizes variation across settings, while the second exposes a different input or consequence. Describe one observation that would support using each system and one observation that would count against it. If your criteria cannot distinguish the cases, they are probably too broad to guide a real decision.

A useful analysis separates description from evaluation. “The system produced this result” is descriptive. “The result is accurate enough, fair enough, or lawful enough to use” is an evaluation that requires a threshold and evidence. Record both statements separately. This keeps a fluent interface, impressive demo, or familiar brand from silently setting the quality bar. This sorting warehouse parcels example is being used here to test variation across settings.

Read

Evidence and people

Use task tests and accountable review as a starting artifact. Include difficult cases, not merely average ones, and note who was absent from the test. Ask how a person discovers an error, how quickly it can be corrected, and whether the fallback works in practice. A correction path that requires expertise, money, or time unavailable to the affected person is not an adequate safeguard. This sorting warehouse parcels example is being used here to test variation across settings.

The social question is equally concrete. Who selected the objective? Whose work produced the data or labels? Who benefits from speed, and who spends time fixing mistakes? In the sorting warehouse parcels example, answer each question with a named role. This turns vague language about “society” into responsibilities that a team can assign and review.

Read

A decision procedure

First, state the intended outcome in one sentence. Second, map input, operation, output, action, and affected person. Third, test the most consequential uncertainty using task tests and accountable review. Fourth, compare the proposed use with a simpler non-AI option. Finally, record a proceed, revise, narrow, or stop decision and the observation that would reopen it. This sorting warehouse parcels example is being used here to test variation across settings.

Apply that sequence to A clinic assigns transcription to software but keeps diagnosis with clinicians. A sensible decision may preserve assistance while removing automatic action, or allow a low-stakes trial while prohibiting higher-stakes use. “Human review” counts only when the reviewer has time, relevant knowledge, access to evidence, and genuine authority to disagree. Otherwise it is a decorative promise rather than a control. This sorting warehouse parcels example is being used here to test variation across settings.

Checking tutor…

Continue learning · glossary & guides
  1. In the sorting warehouse parcels scene, what exactly is the bounded task?
  2. Which piece of task tests and accountable review would most change your decision, and why?
  3. How does the limitation that speed on patterns is not judgment about goals or values affect the quality bar?
  4. Who can correct the output before harm follows, and what authority do they need?
  5. Transfer this page’s lens—variation across settings—to detecting image defects. What stays the same and what changes?

A complete answer distinguishes observation, inference, and value judgment. It also names a threshold rather than saying “be careful,” and it leaves a record another person could challenge. This sorting warehouse parcels example is being used here to test variation across settings.