Page 5 of 8~96 min topic

Prompt caching

Anticipate failure modes

Name failures by their mechanism in prompt caching on the policy-manual assistant, not with a generic hallucination label.

~12 min this pageFailure modes

1Learn the idea

Read

Response design

For each severe prompt caching failure on the policy-manual assistant, define stop condition, safe state, owner, and lasting prevention. Rollback only works if prior prompts, indexes, and models remain available. “Send to a human” needs queue capacity and context—not just a button name.

Run one tabletop on the policy-manual assistant for prompt caching: inject a defect, verify detection, contain, recover, and keep the blameless trace.

Read

Make it operational

After the tabletop, store the injected prompt caching defect for the policy-manual assistant as a regression fixture. If the same failure later reaches users silently, your detection story was aspirational. Detection without a fixture tends to rot for prompt caching.

Also pin one numeric memory from this prompt caching chapter: without caching: 12.3M input tokens/day; with 90% prefix hits: 1.2M uncached prefix + 0.3M suffix = 1.5M full-price-equivalent tokens before cache-read pricing That number is not decoration; it is a template for how claims about prompt caching on the policy-manual assistant should look in design docs. Scoped specifically to prompt caching / policy-manual assistant / failure-modes.

Read

Common mix-ups

People confuse prompt caching with neighboring buzzwords when debugging the policy-manual assistant. Before changing prompts, ask whether the broken stage was evidence gathering, the prompt caching judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried prompt caching and it failed”) that blocks the next team on the policy-manual assistant. Scoped specifically to prompt caching / policy-manual assistant / failure-modes.

Read

Rehearsal (prompt-caching/failure-modes)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt caching rather than generic AI advice.

Read

Rehearsal (prompt-caching/failure-modes)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt caching rather than generic AI advice.

Read

Rehearsal (prompt-caching/failure-modes)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt caching rather than generic AI advice.

Read

Rehearsal (prompt-caching/failure-modes)

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt caching rather than generic AI advice.

Go deeper

Before you start

Why this matters

Invent an incident for the policy-manual assistant involving prompt caching. What earliest signal should fire before users complain?

Prefix twitch

Detect with one whitespace breaks hits. Respond by byte-stable templates.

Stale policy in cache

Detect with doc updated, cache not invalidated. Respond by tie invalidation to doc version.

Caching secrets

Detect with API keys in prefix. Respond by never cache unredacted secrets.

Wrong savings math

Detect with ignoring cache-read price. Respond by use provider’s real tariff.

In the wild

See how this idea shows up as a product and a company — then come back to the lesson. Skills transfer across vendors.

Check your understanding

Page assessment

Answer from memory. Completion is saved from this evidence, not from opening the next page.

1. What is one idea from this page you would apply, and what evidence would you check?

All responses are required.