Page 1 of 8~116 min topic

Why AI makes mistakes

Sounding smart is not the same as knowing

AI can write a sentence that looks perfect and still invent the facts inside it. Smooth language is not proof.

~14 min this pageHook and mental model

1Try it yourself

Simulation game

Hallucination hunt

Stamp each claim: Trap or Trust. Confident voice ≠ true.

Quiz show

SOUNDS SURE

Sydney is the capital of Australia.

2Learn the idea

Read

Guessing dressed up as answers

See it

Why fluent answers can still be wrong
01Predict ≠ lookupSounds like an answer
02Web is messyFacts + fanfic mix
03No embarrassmentCan sound sure
04Prompt trapAsked to invent detail

Confidence is a tone — verify before you act

Language models predict what words usually come next. That is useful for drafts, outlines, and brainstorming. It is dangerous when you treat the output like a researched fact.

A made-up book title is a classic “hallucination”: the answer matches the shape of a good reply (author + title + summary) without matching the real world. The model is not lying on purpose. It is completing a pattern.

Try this separation every time:

  1. Fluency — Does it read clearly?
  2. Claim — What exact fact is being asserted?
  3. Check — Where could you confirm it in under two minutes?

If you cannot name a check, do not paste the claim into homework.

Read

Why kids get fooled

School rewards clean paragraphs. AI is great at clean paragraphs. Your brain hears “this sounds like a teacher answer” and relaxes. That is the trap.

Specific details make the trap worse. A fake ISBN, a fake quote, or a fake year can feel more trustworthy because they look researched. Specificity is not evidence. Evidence is a source you can open.

Read

A homework habit

Before you use an AI book list, sports fact, or historical date:

  • Ask for sources and verify them yourself.
  • Prefer your teacher’s reading list, the library catalog, or a textbook.
  • Keep AI for structure (“help me outline themes”) while you supply the real titles.

When something sounds too neat — perfect titles, perfect quotes — slow down. Neatness is cheap. Truth takes a check.

Read

What prediction is doing

Imagine autocomplete on your phone, but for whole paragraphs. The system has seen millions of sentences about books, so it can invent a title that fits the pattern of real titles. That is prediction. Knowledge would mean the title maps to a shelf, an ISBN, or a library record you can open.

This difference matters outside English class too. Game tip articles, “fun facts” for morning announcements, and science fair background blurbs all reward tidy wording. If you only reward tidy wording, you will ship invented details.

A helpful reframe: treat AI like a classmate who talks fast and hates saying “I don’t know.” You would not put that classmate’s unverified book list in your Works Cited. Give the same courtesy to a chatbot.

Read

Practice line you can reuse

“Help me brainstorm search terms and questions. Do not invent titles, quotes, authors, or page numbers. If you are not sure something exists, say so.”

Put that sentence at the top of homework chats for a week. Watch how often the model still tries to sound complete — and how often your library check saves you.

Go deeper

Before you start

Why this matters

You ask a chatbot for three books about friendship for your English essay. It lists a title that sounds real — The Midnight Bench by “A. Rivera” — and even summarizes a plot. Your library search finds nothing. The book never existed.

In the wild

See how this idea shows up as a product and a company — then come back to the lesson. Skills transfer across vendors.

Check your understanding

Page assessment

Answer from memory. Completion is saved from this evidence, not from opening the next page.

1. Why can a fluent AI answer still be wrong about a book title?
2. What is the difference between a “likely sentence” and a “checked fact”?
3. Name one homework claim you should never trust without a source.
4. What two-minute check would catch a fake book title?
5. How would you use AI for an essay *without* treating it as a research database?

All responses are required.