Mastery: connect the pieces

1. Idea — Prompt caching reuses computation for an identical stable prefix (system prompt, shared docs). It is not caching the final answer string.…
2. Mechanism — Pin a stable prefix, enable provider or explicit cache, send only varying suffixes, bill/read cache hits differently, invalidate when prefix bytes change.…
3. Controls — prefix length, cache TTL, explicit cache keys, suffix-only inputs, invalidation on doc edit
4. Tradeoff — A longer cached prefix can save more input work but may carry irrelevant context and increase uncached misses when any early token changes. Explicit caches impr…
5. Failures — Prefix twitch; Stale policy in cache
6. Metrics — cache hit rate, $ /day input, p5 time-to-first-token, invalidation lag
7. Ship rule — Cache the 12k policy prefix after hit rate ≥90% on canary and invalidation <5 minutes after edits.

Neighboring layers (retrieval, serving, policy, human review) still own what prompt caching cannot on the policy-manual assistant. Do not ask this chapter’s dial to replace permissions or source truth.

Mastery is transfer: take prompt caching into a second scenario that is not the policy-manual assistant and rebuild the checklist without copying sentences. If you can only recite this chapter’s examples for prompt caching, you have memorized a story, not a model.

Also pin one numeric memory from this prompt caching chapter: without caching: 12.3M input tokens/day; with 90% prefix hits: 1.2M uncached prefix + 0.3M suffix = 1.5M full-price-equivalent tokens before cache-read pricing That number is not decoration; it is a template for how claims about prompt caching on the policy-manual assistant should look in design docs. Scoped specifically to prompt caching / policy-manual assistant / mastery-connect.

People confuse prompt caching with neighboring buzzwords when debugging the policy-manual assistant. Before changing prompts, ask whether the broken stage was evidence gathering, the prompt caching judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried prompt caching and it failed”) that blocks the next team on the policy-manual assistant. Scoped specifically to prompt caching / policy-manual assistant / mastery-connect.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt caching rather than generic AI advice.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt caching rather than generic AI advice.