Build the mental model

Think of keeping a pre-set stage for a play so each night you only rehearse the new scenes. Use the analogy to name the moving parts for prompt caching, then drop it when you need numbers. For the policy-manual assistant, the enduring idea is not a vendor feature name; it is the decision prompt caching changes and the evidence that decision leaves behind.

Prompt caching reuses computation for an identical stable prefix (system prompt, shared docs). It is not caching the final answer string.

Beginners often blur neighboring ideas when discussing prompt caching. Keep it distinct by asking what artifact would still exist if model weights were frozen and only this layer changed on the policy-manual assistant. If you cannot name that artifact, you are still describing “the AI” in general.

Pin a stable prefix, enable provider or explicit cache, send only varying suffixes, bill/read cache hits differently, invalidate when prefix bytes change. In day-to-day language for prompt caching: someone brings a need, the system inspects allowed evidence, this layer contributes a judgment or structure, and a consequence reaches a user or downstream system. Deterministic guards—permissions, schemas, arithmetic—still belong to the application around the policy-manual assistant.

Uncertainty is normal for prompt caching. Incomplete inputs and probabilistic behavior mean the policy-manual assistant needs an escape hatch (retry, fallback, escalate) rather than fake certainty in fluent prose.

When you explain prompt caching to a new teammate on the policy-manual assistant, forbid the sentence “the AI just knows.” Replace it with the artifact that moves and the evidence you would file for prompt caching. If they can falsify your picture with a single counterexample from last week’s traffic on the policy-manual assistant, your mental model is working.

Also pin one numeric memory from this prompt caching chapter: without caching: 12.3M input tokens/day; with 90% prefix hits: 1.2M uncached prefix + 0.3M suffix = 1.5M full-price-equivalent tokens before cache-read pricing That number is not decoration; it is a template for how claims about prompt caching on the policy-manual assistant should look in design docs. Scoped specifically to prompt caching / policy-manual assistant / mental-model.

People confuse prompt caching with neighboring buzzwords when debugging the policy-manual assistant. Before changing prompts, ask whether the broken stage was evidence gathering, the prompt caching judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried prompt caching and it failed”) that blocks the next team on the policy-manual assistant. Scoped specifically to prompt caching / policy-manual assistant / mental-model.