Worked case: investigate a hallucination

A disciplined investigation preserves the original claim, identifies its decision impact, and checks independent primary evidence before rewriting the summary.

Use this three-part method:

1. Extract each checkable claim. Write down what this means in the scenario, what evidence would show it was done, and who owns the decision.
2. Trace or reproduce the alleged evidence. Write down what this means in the scenario, what evidence would show it was done, and who owns the decision.
3. Record confirmed, contradicted, and unresolved findings. Write down what this means in the scenario, what evidence would show it was done, and who owns the decision.

Walk through A manager receives a market summary claiming a competitor launched a product last quarter and must decide whether to change a roadmap.. Label three moments where “Worked case: investigate a hallucination” changes what you trust: (1) the first fluent answer, (2) the first missing source or permission, and (3) the decision a human must own. Write the before/after task so the model only does the slice that evidence supports. Keep one sentence that states how this page’s idea differs from a generic “AI is smart/dumb” score.

• Quietly editing the claim with no audit trail. This shortcut removes useful friction, but it also hides an assumption that should be tested. Replace it with an observable check.
• Searching only for pages that agree. This shortcut removes useful friction, but it also hides an assumption that should be tested. Replace it with an observable check.
• Turning one discovered error into a guess about every other claim. This shortcut removes useful friction, but it also hides an assumption that should be tested. Replace it with an observable check.

Before you close the tab, capture a reusable habit for Worked case: investigate a hallucination inside Why AI makes mistakes: name the observable check, the evidence you would open, and the stop condition. Rehearse it once on a low-stakes example, then once on a higher-stakes variant. The habit succeeds when you can explain the check without reopening this lesson. Target outcome: Explain why plausible language generation does not guarantee factual accuracy.

Return to the why ai makes mistakes scenario and restate what this chapter proved on page worked hallucination investigation.mdx. Name one metric, one ownership rule, and one regression test you will keep. Explain how this page connects to the previous page without repeating earlier paragraphs. If you cannot name a falsifier, the chapter is still a story rather than a controlled practice. Write the falsifier as an observable event with a threshold.

Return to the why ai makes mistakes scenario and restate what this chapter proved on page worked hallucination investigation.mdx. Name one metric, one ownership rule, and one regression test you will keep. Explain how this page connects to the previous page without repeating earlier paragraphs. If you cannot name a falsifier, the chapter is still a story rather than a controlled practice. Write the falsifier as an observable event with a threshold.