What gets cached
Providers can reuse processing for eligible repeated prompt content, often a stable prefix such as a system prompt, tool definitions or a document. Cache behavior differs by API. OpenAI documents automatic caching for eligible prompts; Anthropic documents explicit cache controls and separate write/read charges. Read the rules for the exact model and endpoint.
Calculate the effective rate
Let an input segment contain N tokens. If its normal rate is R and a cache hit rate is H with cached-read rate C, the simple expected cost is N × ((1 − H)R + HC) / 1,000,000. If cache writes have a premium, add that cost. The formula shows why a large advertised discount does not equal the same reduction in the whole bill: output and uncached input still cost money.
A worked illustration
Suppose a repeated 10,000-token prefix has a $3 standard input rate, a $0.30 cached-read rate and an 80% cache hit rate. Per request, expected prefix cost is 10,000 × (0.2 × 3 + 0.8 × 0.30) / 1,000,000 = $0.0084, versus $0.03 without caching. That is 72% less for this prefix alone, before cache writes and output. The rates are illustrative.
Measure in production
Log cache creation and read tokens from usage records. Keep the repeated prefix stable and place changing user data later where the API's rules allow. Review expiration, minimum cacheable length and provider-specific retention terms. Compare total invoice before and after, not only one line item.