Prompt caching

Prompt caching lets a provider reuse the processed form of a prompt prefix across requests, billing cached input at a reduced rate.

It is the highest-leverage optimisation available for most production workloads, because the typical application sends the same long system prompt and the same few-shot examples on every single request.

To benefit:

• Put stable content first. The cache matches on a prefix, so anything variable at the start destroys the hit for everything after it.
• Keep the prefix byte-identical. A timestamp or a user ID injected early is enough to miss.
• Watch the reported cache hit rate. A change that silently breaks caching looks like an unexplained price rise.

Losing a cache hit is a common cause of a step change in spend with no change in traffic — see unexpected charges.