Trace a worked example

Goal: improve the policy-manual assistant using prompt caching without breaking protected slices.

Mechanism reminder for prompt caching: Pin a stable prefix, enable provider or explicit cache, send only varying suffixes, bill/read cache hits differently, invalidate when prefix bytes change.

Baseline and shock: without caching: 12.3M input tokens/day; with 90% prefix hits: 1.2M uncached prefix + 0.3M suffix = 1.5M full-price-equivalent tokens before cache-read pricing

Tradeoff in play for prompt caching: A longer cached prefix can save more input work but may carry irrelevant context and increase uncached misses when any early token changes. Explicit caches improve control but require lifecycle management.

Ship decision: Cache the 12k policy prefix after hit rate ≥90% on canary and invalidation <5 minutes after edits.

Rollback triggers for prompt caching must cite cache hit rate, $ /day input, p5 time-to-first-token, invalidation lag. If you cannot name a tolerated regression on the policy-manual assistant, do not promote the change.

Keep the prompt caching decision record beside the policy-manual assistant code paths that implement it. Future you will not remember why a default exists unless the evidence is linked from the config for prompt caching.

Also pin one numeric memory from this prompt caching chapter: without caching: 12.3M input tokens/day; with 90% prefix hits: 1.2M uncached prefix + 0.3M suffix = 1.5M full-price-equivalent tokens before cache-read pricing That number is not decoration; it is a template for how claims about prompt caching on the policy-manual assistant should look in design docs. Scoped specifically to prompt caching / policy-manual assistant / worked-trace.

People confuse prompt caching with neighboring buzzwords when debugging the policy-manual assistant. Before changing prompts, ask whether the broken stage was evidence gathering, the prompt caching judgment itself, validation, or the product action. Fixing the wrong stage creates folklore (“we tried prompt caching and it failed”) that blocks the next team on the policy-manual assistant. Scoped specifically to prompt caching / policy-manual assistant / worked-trace.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt caching rather than generic AI advice.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt caching rather than generic AI advice.

Write a five-line artifact for this page: goal, inputs, check, owner, stop rule. Invent one fluent failure that the check would catch. Keep details specific to prompt caching rather than generic AI advice.

For prompt caching, add acceptance test 1: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 2: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 3: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 4: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 5: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 6: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 7: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 8: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 9: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 10: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 11: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 12: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 13: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 14: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 15: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 16: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 17: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 18: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 19: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 20: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 21: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 22: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 23: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 24: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 25: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 26: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 27: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 28: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 29: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 30: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 31: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 32: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 33: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 34: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.

For prompt caching, add acceptance test 35: evidence source, threshold, and signer. Keep it unique to this chapter's scenario.