When Does Prompt Caching Actually Save Money on AI APIs?

Prompt caching is a cost-control decision, not an automatic discount. It pays when a long, stable input prefix is reused enough times before its cache lifetime ends, and when the provider actually reports cache reads. The first write may cost more than an ordinary input pass, while a miss can leave the whole prefix at the regular input rate. This article gives a break-even test that works across providers without assuming that one provider’s cache contract applies to another.

Last reviewed: 2026-08-29

Direct answer

Approve prompt caching only after the expected cache reads outweigh the write premium, the prefix meets the model’s minimum, and a canary shows real hits. Separate the reusable prefix (P) from the changing suffix (S), count requests that occur inside the cache lifetime (N), and use the provider’s uncached input rate (u), cache-write multiplier (w), and cache-read multiplier (r). For a simple one-write, full-hit scenario:

uncached_cost = N * (P + S) * u
cached_cost   = P * (w + (N - 1) * r) * u + N * S * u
approve when  cached_cost < uncached_cost

This is a planning model, not a promise of savings. Add the cost of misses, extra writes, retries, and any provider-specific cache object or storage charge if the contract includes one. A cache read never makes a changing suffix free, and prompt caching does not replace a quality check.

The OpenAI prompt caching guide says supported models can reuse a matching prefix, with cached-input discounts of up to 90 percent, and that caching is enabled by default for supported models. It points operators to a dashboard for cache-read hit rates. Those statements do not establish one universal multiplier for every route. The Anthropic Claude prompt caching documentation supplies a concrete contract example: for its documented five-minute option, writes are 1.25 times the base input rate and reads are 0.1 times that rate; it also offers a one-hour option with a higher write multiplier. Its minimum token count varies by model.

Here is an illustrative calculation using Anthropic’s documented five-minute multipliers, with no dollar price assumed. Suppose P is 4,000 tokens, S is 400 tokens, and ten requests share the prefix while the cache remains live. Ten uncached passes consume 44,000 input-token units. One write plus nine full reads consumes 4,000 * (1.25 + 9 * 0.1) + 10 * 400 = 12,600 normalized units. The apparent saving is large because the example assumes every later request hits. If only two requests arrive, or if the prefix changes and misses, the result is different. Replace the normalized units with the actual model rate and observed counters before approving a budget.

Google’s Gemini context caching guide says implicit caching is enabled by default on Gemini 2.5 and newer models, but the minimum input is model-specific: the listed Gemini 2.5 models require 2,048 tokens, while the listed Gemini 3.x models require 4,096. Google recommends placing common content at the beginning and sending similar prefixes close together, then exposes total_cached_tokens in usage. Amazon’s Bedrock prompt caching guide likewise warns that an eligible prompt does not guarantee a hit, and says cached and written token fields must be read from the response. These caveats make measured hit rate more important than a spreadsheet-only forecast.

Who this is for

This guide is for:

• AI API budget owners deciding whether a cache feature belongs in a forecast.
• Platform engineers shaping system instructions, tools, retrieval context, and user content.
• FinOps analysts reconciling provider usage with a gateway ledger.
• Reliability or operations leads who need a safe canary and a clear rollback condition.

It is not a substitute for the price sheet or model card for the exact provider, model, region, API, and retention option you will use. Treat the formulas as a common measurement frame, then fill them with the selected contract’s rates and limits.

Key takeaways

• Put stable instructions, tool definitions, examples, and reference context before dynamic user data. Exact-prefix matching means a timestamp or changing block before the breakpoint can invalidate the reusable prefix.
• Count the prefix with the model and endpoint you will run. A prefix below the documented minimum may be processed normally even when a cache marker is present.
• Model the first write separately from subsequent reads. A write premium can make a one-off request more expensive; savings generally require reuse inside the TTL window.
• Measure cache_read_tokens, cache_write_tokens, or the provider’s equivalent. A configured breakpoint is not evidence that a read occurred.
• Keep the uncached suffix, output tokens, retries, and misses in the ledger. Caching the prefix does not eliminate those costs.
• Roll out by workload and model version. A cache contract, TTL, or minimum can differ across providers and models, so do not copy a threshold from one row of a budget sheet to another.

Sources checked

These public provider guides underpin the factual claims and give you a place to verify the contract for your route:

• OpenAI, Prompt caching — prefix reuse, cached-input discount, model minimums, write/read accounting, retention, and usage fields.
• Anthropic, Prompt caching — automatic and explicit breakpoints, exact matching, five-minute and one-hour TTLs, model thresholds, and cache counters.
• Google AI for Developers, Context caching — implicit caching defaults, model-specific minimums, prefix-order guidance, and total_cached_tokens.
• Amazon Web Services, Prompt caching for faster model inference — implicit versus explicit behavior, checkpoint limits, TTLs, billing fields, best-effort hits, and endpoint restrictions.

The arithmetic and workflow below are operator guidance derived from those documented fields. They are not a claim that every provider uses the same multiplier or guarantees the same hit rate.

Contract details to verify

Before changing a budget, record the following contract facts for the exact route:

Do not infer a guaranteed discount from a feature flag. For example, Bedrock’s documentation says a repeated identical prompt can still miss, while Anthropic’s documentation says changing any content at or before a breakpoint changes the prefix identity. Verify the response counters and the invoice mapping after a canary.

Operator workflow: happy path and error path

1. Baseline. Choose one workload with a clearly repeatable prefix. For a small canary, capture enough requests to include both the first write and several expected repeats. Record model, endpoint, region, prompt version, prefix token count, suffix token count, output token count, latency, status, and provider-reported cost. Do not log prompt text or credentials; use a short fingerprint for correlation.

2. Shape the request. Put static system instructions, tools, examples, and retrieved material first. Put the user question, timestamp, tenant-specific facts, and other changing values last. Decide whether an explicit breakpoint is needed. If you use automatic caching, confirm where the provider will place the breakpoint and how far it searches for an earlier entry.

3. Check the contract. Confirm the model minimum, write and read rates, TTL, breakpoint limit, and endpoint support from the source for that route. For Gemini, check the model’s listed minimum; for Bedrock, check the model card and whether the endpoint is on-demand rather than batch. Store these assumptions beside the budget version.

4. Run the happy path. Send the first request, wait until the provider has begun processing it, then send matching requests within the documented lifetime. A healthy trace shows a write on the first eligible request and positive read counters on later matching requests. Keep a quality sample so a lower token bill is not mistaken for a successful product result.

{
  "request_id": "req-demo-01",
  "workload": "support-summary",
  "model": "provider-model-id",
  "route": "native-gateway",
  "prompt_version": "v3",
  "prefix_fingerprint": "hash-demo",
  "prefix_tokens": 4000,
  "suffix_tokens": 400,
  "cache_mode": "explicit",
  "cache_ttl": "30m",
  "cache_read_tokens": 3600,
  "cache_write_tokens": 0,
  "input_tokens": 400,
  "output_tokens": 180,
  "cache_hit": true,
  "status": "success",
  "error_code": null
}

These are sanitized logging fields, not a provider-specific schema. Keep the fingerprint non-reversible and omit prompt bodies, authorization headers, account identifiers, and secret material.

5. Calculate realized cost. Sum cache-write tokens at the write rate, cache-read tokens at the read rate, ordinary input tokens at the base rate, and output tokens at the output rate. Follow the selected API’s field definitions so cached input is neither omitted nor counted twice. Compare the result with the baseline and report hit rate as cached tokens divided by total input tokens, or use the provider’s documented metric.

6. Decide or roll back. Promote only if the measured margin remains positive after misses, TTL expiry, retries, and quality checks. If the happy path fails, follow this error path: first classify the response as below minimum, prefix mismatch, expired entry, unsupported endpoint, or unavailable usage counter; then remove the explicit marker or return an implicit-cache canary to its baseline route, preserve the baseline, and open a contract check. Do not keep retrying blindly, because retries can create more writes without creating reads.

Failure modes

• One-shot traffic. A first write is the only event, so its premium can exceed the cost of an uncached pass. Keep one-shot jobs out of a cache forecast.
• Prefix below minimum. Providers document minimum token counts by model. The request may succeed while the prefix is simply not cached; treat zero read and zero write counters as a signal to investigate.
• Mutable content before the breakpoint. A changing date, user identifier, tool schema, or retrieved passage changes the exact prefix. Move the breakpoint after the last stable block or split stable sections deliberately.
• TTL expiry. A five-minute or thirty-minute window is measured by the provider’s rules, and long generation time can consume part of it. If requests arrive less often than the lifetime, model a new write for each active period.
• Best-effort misses. Both implicit and explicit features can be eligible without guaranteeing a hit. Use observed counters, not a feature toggle, as the billing truth.
• Unsupported endpoint. Bedrock documents prompt caching as unavailable for batch inference. A route migration can silently remove the expected cache behavior unless endpoint type is a ledger field.
• Breakpoint limits and lookback gaps. Anthropic documents up to four explicit breakpoints and a finite lookback for automatic caching. Too many markers or a growing conversation can produce a miss or a documented request error.
• Cross-region or model changes. Bedrock notes that cross-region inference can increase cache writes during demand spikes. A model-version or region change can also alter minimums and retention, so start a new baseline.
• Misread usage fields. Field names and accounting boundaries differ across APIs. Map ordinary input, cache reads, and cache writes from the selected provider’s documentation before reconciling an invoice.

FAQ

Does prompt caching always save money? No. It is a conditional optimization. Reuse count, prefix length, write premium, read discount, suffix size, TTL, and actual hit rate determine the result.

Should the user message be cached? Usually keep per-request text after the stable breakpoint. This preserves an exact reusable prefix while allowing the user message to change. Confirm the provider’s block ordering and breakpoint rules first.

How many requests are needed to break even? There is no universal number. Under Anthropic’s documented five-minute multipliers, one write plus one full read costs 1.35 times one prefix pass, versus two ordinary passes; a single request is not cheaper. Add misses, suffix tokens, and the model’s actual rates to your calculation.

Can I rely on an implicit cache without code changes? Some providers enable implicit caching by default, including Gemini 2.5 and newer according to its guide. “Enabled” still means best effort, so inspect usage counters and keep a baseline.

What if the response has no cache fields? Treat the contract as unverified. Pause the rollout, check the exact API and SDK response shape, and reconcile provider billing before claiming savings.

Does caching reduce output cost or guarantee a quality result? The documented feature targets reusable input prefixes. Output generation and application quality still need their normal budget and evaluation checks; do not remove those lines from the forecast.

Reader next step

Create a small ledger today with these columns: workload, model, route, prompt version, prefix tokens, suffix tokens, cache TTL, ordinary input tokens, cache-read tokens, cache-write tokens, output tokens, hit status, latency, and realized cost. Run a baseline, then a cache-enabled canary with the same prefix and a deliberately changing suffix. Approve the rollout only when the observed formula stays below the baseline for the traffic pattern you actually expect.

For gateway-level accounting, continue with Trace CometAPI Cost and Usage for Token Budgets . If the canary produces misses or unexpected charges, use Review CometAPI Error and Cost Signals Before Token Budget Decisions to organize the evidence before changing the budget.