Reference · Glossary

Prompt caching

Reusing **cached prefix tokens** (system prompt, long docs) so repeat calls pay less for the same context.

#When to use

Stable system prompts and long shared context in high-volume APIs.

When every request has a unique megabyte context — cache hit rate stays low.

Same 2k-token policy block on every chat → cache hits skip re-billing that prefix.