Chapter BSpot wrong answersPage 2 of 8

Spot wrong answers

Map checkable claims

Verification starts by turning an answer into atomic claims with different evidence needs.

~14 minCore mental model

Before you start

Why this matters

A travel answer mixes opening hours, journey times, prices, safety advice, and subjective recommendations in one polished paragraph. The temptation is to begin by typing a broad request and judging whatever appears. That approach makes a good result hard to repeat and a bad result hard to diagnose.

Instead, pause before generating. Ask what evidence the output may use, what decision it supports, who will review it, and what happens if it is wrong. These questions turn AI from an impressive text box into one component of a controlled workflow. They also reveal when another tool or a qualified person is needed.

1Learn the idea

Read

The working principle

Verification starts by turning an answer into atomic claims with different evidence needs. This principle matters because an AI system produces likely output from the context and instructions it receives; it does not automatically know the organization’s current facts, private policy, unstated intent, or acceptable risk.

Good work therefore has two layers. The content layer contains the draft, labels, plan, extraction, or analysis. The control layer contains source boundaries, acceptance criteria, uncertainty rules, permissions, and review. Beginners often focus only on content because it is visible. Reliable users design both layers.

Use the following sequence for this page: underline names, dates, numbers, quotations, and causal claims; separate facts from judgments and proposals; rank claims by consequence and uncertainty; assign each important claim a verification method. The sequence is a guide, not a ritual. Skip a step only when its question truly has no effect on the outcome, and strengthen it when mistakes would be costly.

Read

A practical method

1. Underline names, dates, numbers, quotations, and causal claims

Make this step visible in the working material rather than leaving it as an assumption. Write down what is known, what is supplied by the user or source, and what remains undecided. If the result cannot be reviewed against this step, tighten the instruction or add a checkpoint. The aim is not to make the prompt longer; it is to make the work observable and correctable.

2. Separate facts from judgments and proposals

Make this step visible in the working material rather than leaving it as an assumption. Write down what is known, what is supplied by the user or source, and what remains undecided. If the result cannot be reviewed against this step, tighten the instruction or add a checkpoint. The aim is not to make the prompt longer; it is to make the work observable and correctable.

3. Rank claims by consequence and uncertainty

Make this step visible in the working material rather than leaving it as an assumption. Write down what is known, what is supplied by the user or source, and what remains undecided. If the result cannot be reviewed against this step, tighten the instruction or add a checkpoint. The aim is not to make the prompt longer; it is to make the work observable and correctable.

4. Assign each important claim a verification method

Make this step visible in the working material rather than leaving it as an assumption. Write down what is known, what is supplied by the user or source, and what remains undecided. If the result cannot be reviewed against this step, tighten the instruction or add a checkpoint. The aim is not to make the prompt longer; it is to make the work observable and correctable.

Read

Work through the scenario

Return to the opening case: A travel answer mixes opening hours, journey times, prices, safety advice, and subjective recommendations in one polished paragraph. Begin by rewriting the request as a small contract. Name the intended reader or user, the authoritative material, the operation to perform, the required output, and the review owner. If current information is required, identify where it will come from. If exact calculation or action is required, assign that step to a deterministic tool or an approved system rather than relying on prose generation.

A useful instruction could follow this shape:

Goal: help [reader] accomplish [outcome]. Use only [named sources or supplied material] for factual claims. Perform [specific operation] and return [format]. Mark missing information as TBD or ask a focused question; do not guess. Before the result is used, [named person or role] will check [criteria].

The brackets are not decoration. Each placeholder forces the operator to make a decision. If a field is unknown, that is information about the workflow. Do not let the model silently fill it.

Now test the result with one ordinary case and one difficult case. The difficult case should contain something realistic: missing data, contradictory dates, a long input, an unusual category, a sensitive field, or an instruction that conflicts with the main goal. Compare both outputs with the same criteria. A pattern that works only on the clean example is not ready for routine use.

Read

Failure modes to catch

  • Checking only the easiest claim. This hides an important assumption or removes a review point. Replace it with an explicit rule, a source check, or a human decision.
  • Treating several linked assertions as one fact. This hides an important assumption or removes a review point. Replace it with an explicit rule, a source check, or a human decision.
  • Spending equal effort on harmless preferences and safety-critical details. This hides an important assumption or removes a review point. Replace it with an explicit rule, a source check, or a human decision.

Another common failure is automation bias: once a neat answer exists, reviewers search for reasons to accept it. Counter that tendency by checking the source first, then the wording. For important work, ask a reviewer to inspect the output without seeing the model’s confidence language. Confidence should come from evidence and tests, not tone.

Keep the review proportional. A low-stakes brainstorming list may need only a quick relevance scan. A message to a customer needs fact, tone, and authorization checks. Advice affecting safety, rights, employment, money, or legal obligations may need a qualified professional and authoritative current sources. A longer prompt does not reduce that responsibility.

Read

Make it reusable

After a successful run, save the pattern without the original sensitive data. Record its purpose, required inputs, prohibited uses, expected output, and review checklist. Add two or three representative test cases. Date the template and name an owner if others depend on it.

When it fails, diagnose the failure before editing. Was the source missing, the task ambiguous, the format weak, the tool unsuitable, or the review skipped? Change one important element and rerun the same test. This produces a useful lesson; random prompt expansion usually produces only a longer prompt.

Checking tutor…

Continue learning · glossary & guides