Reference · Glossary
Prompt caching
Reusing **cached prefix tokens** (system prompt, long docs) so repeat calls pay less for the same context.
When to use
Stable system prompts and long shared context in high-volume APIs.
When not to
When every request has a unique megabyte context — cache hit rate stays low.
Example
Same 2k-token policy block on every chat → cache hits skip re-billing that prefix.