Understand the mechanism

Pin a stable prefix, enable provider or explicit cache, send only varying suffixes, bill/read cache hits differently, invalidate when prefix bytes change.

Read the prompt caching path as a pipeline for the policy-manual assistant. At each stage, name the representation, the owner, and how information can be lost. Identifiers must mark prompt versions, model versions, indexes, and policies so “randomness” is not the default explanation for every bug in prompt caching.

without caching: 12.3M input tokens/day; with 90% prefix hits: 1.2M uncached prefix + 0.3M suffix = 1.5M full-price-equivalent tokens before cache-read pricing Scoped specifically to prompt caching / policy-manual assistant / mechanism.

Keep the unit and the denominator visible when you discuss prompt caching. A percentage without a base, or a latency without a percentile, hides the failure mode this chapter cares about on the policy-manual assistant.

Learned stages estimate; deterministic stages enforce. A fluent result from the policy-manual assistant does not prove prompt caching used the right evidence. Preserve intermediates when privacy allows—candidate lists, traces, scores, citations—so you can see the first broken stage in the prompt caching path.

Operational correctness for prompt caching includes deadlines on the policy-manual assistant. If stages that feed this workload sum past the user’s patience, trim earlier—usually pack less, retrieve less, or parallelize—before blaming the model vendor for prompt caching. Mechanism diagrams that ignore time are incomplete.

Also pin one numeric memory from this prompt caching chapter: without caching: 12.3M input tokens/day; with 90% prefix hits: 1.2M uncached prefix + 0.3M suffix = 1.5M full-price-equivalent tokens before cache-read pricing That number is not decoration; it is a template for how claims about prompt caching on the policy-manual assistant should look in design docs. Scoped specifically to prompt caching / policy-manual assistant / mechanism.

People confuse prompt caching with neighboring buzzwords when debugging the policy-manual assistant. Before changing prompts, ask whether the broken stage was evidence gathering, the prompt caching judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried prompt caching and it failed”) that blocks the next team on the policy-manual assistant. Scoped specifically to prompt caching / policy-manual assistant / mechanism.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt caching rather than generic AI advice.