Tutorials · Chapter C (3/4) · ~9 min
Transformers in plain English
Try it → see it → read → next
Attention = “which earlier words matter for the next word?”
Try yourself
Playground
Attention heatmap
Tap a word. Brighter neighbors are what the model “looks at” more.
“Bank” leans on “river” — water bank, not money.
Try focusing bank vs river — watch the glow jump.
Recap
What you just did
You mapped meanings. Transformers are meaning-routing machines for sequences.
Teach
How it works
See it
Hot tokens = higher attention when guessing what comes next
The model weighs nearby words to decide the next piece
See it: hot tokens = stronger attention. When you type “bank,” earlier words like “river” or “money” change what “bank” should mean — weighted focus, not feelings.
Chain: tokens → vectors → attention mixes neighbors → predict next token. Stack that deep → grammar, style, and fact-ish patterns (still not a truth machine).
Use it
When you'd use this
- Reading model cards (“decoder-only transformer”).
- Understanding why long context helps (more tokens to attend over) and costs more.
Watch out
Watch out
“Transformer” ≠ Optimus Prime, and ≠ automatic truth. It’s a pattern architecture, not a guarantee.
Try next
Try this next
Use the Try yourself playground above first — click around until the win banner appears.
Explain attention to a friend using the river bank / money bank example in one sentence.