Amazon Bedrock Prompt Caching: Real Savings, Real Limits
According to the Amazon Machine Learning blog, Amazon Bedrock now offers prompt caching – a feature that can shave up to 90 % off the cost of input tokens when the same context is sent repeatedly. For teams that run Retrieval‑Augmented Generation (RAG) or other multi‑question flows, the claim promises cheaper and faster responses without changing the prompt itself.
How Prompt Caching Works
Prompt caching stores a snapshot of the part of a request that has already been processed. When a request contains a special cachePoint marker, Bedrock checks whether the tokens before that marker match a cached entry. If they do (a cache hit), the model skips re‑tokenising and re‑computing those tokens and starts generation from the cached state. If they don’t (a cache miss), the model processes the whole input and writes the result to the cache for future hits.
Four knobs control the behaviour:
- Cache scope – entries live only within the AWS account and Region that created them.
- Token thresholds – a cache checkpoint must contain a minimum number of tokens (1,024 for Claude Sonnet 4.5/4.6, 4,096 for Opus models) before Bedrock will consider it cacheable.
- TTL (time‑to‑live) – how long an entry stays valid. The default is 5 minutes; some models allow up to 1 hour.
- Model‑agnostic syntax – the
cachePointJSON shape is identical for Anthropic Claude, Amazon Nova, and other supported families.
In practice, you place the marker right after the static part of a prompt – for example a long policy document – and before the variable question. The first request writes the document to cache (incurring a higher write price). Subsequent requests read the cached tokens, which are billed at a dramatically reduced rate.
Practical Scenarios Tested
The blog walks through six scenarios; the two most common for office‑level workloads are:
- Message content caching – a 10,000‑token contract is cached once, then ten follow‑up questions are asked. The first call pays a cacheWriteInputTokens price (25 % higher than a normal input token). Each of the nine follow‑ups pays cacheReadInputTokens, which is 90 % cheaper than a standard input token.
- System prompt caching – a persona definition or tool schema is cached across many conversations. Because the cached portion never changes, every new user query benefits from a cache hit, reducing both latency (time‑to‑first‑token) and cost.
Both patterns require the static block to exceed the model’s token threshold; otherwise Bedrock falls back to a normal request.
Cost and Latency Impact
Below is a simplified cost comparison based on the pricing table in the post. Numbers are shown as a multiple of the standard input‑token price; actual dollars depend on the model you use.
| Token type | Billing multiplier vs. standard input |
|---|---|
| Standard input | 1.0 |
| cacheWriteInputTokens | 1.25 |
| cacheReadInputTokens | 0.10 |
| cacheWriteInputTokens (1‑hour TTL) | 2.0 |
Example calculation – a 10,000‑token document plus ten 200‑token questions:
First request (write): 10,000 × 1.25 = 12,500 token‑units. Nine reads: 200 × 0.10 × 9 = 180 token‑units. Total: 12,680 token‑units versus 10,000 × 1 + 200 × 10 = 12,000 token‑units without caching.
The net saving is roughly 75 % on input‑token cost for the repeated‑context portion, assuming all reads happen before the TTL expires. Latency drops as well because the model skips the heavy upfront computation; the blog reports a noticeable reduction in time‑to‑first‑token (TTFT).
What Changes and the Trade‑offs Nobody Highlights
The headline benefit – cheaper, faster repeats – comes with hidden constraints.
- Cache warm‑up cost – The first request is more expensive (25 % higher, or double for a 1‑hour TTL). If your workload only asks a single question per document, you pay extra without any return.
- Token‑threshold barrier – Small prompts (e.g., a 300‑token FAQ) never hit the cache, so teams that split documents into chunks below the threshold see no savings.
- TTL management – A short TTL (5 minutes) protects stale data but forces frequent rewrites for long‑running sessions. Extending TTL to an hour doubles the write price and may keep stale context around longer than desired.
- Region‑scoped caches – Multi‑Region deployments duplicate caches, effectively resetting the write cost each time traffic hops regions. That can erode the expected 75 % saving.
- Cache‑hit predictability – The cache only helps when the prefix is identical. Changing even a single token – a stray space or a different date format – forces a miss and re‑writes the cache.
In practice, the sweet spot is a stable, sizable static block (≥ 1k tokens) that is reused many times within the same region and TTL window. Anything else either sees little cost reduction or pays a higher write price for minimal benefit.
Quick‑Start Checklist for Your Team
- Identify repeatable context – documents, persona prompts, or tool schemas that exceed the model’s token threshold.
- Insert a
cachePointmarker right after that static block in the Converse API payload. - Set an appropriate TTL – 5 minutes for short‑lived sessions, 1 hour only if you expect heavy reuse and can tolerate the higher write price.
- Monitor hit‑rate – log whether each request is a cache hit or miss; aim for > 80 % hit‑rate to justify the feature.
- Validate latency – measure TTFT before and after caching; a noticeable drop confirms the cache is being used.
- Scope to one region – keep your Bedrock calls in a single AWS Region to avoid unnecessary cache rewrites.
Try it today: take a 2‑page policy PDF, paste its text into a Bedrock request, add a cachePoint after the last paragraph, and fire off three follow‑up questions within ten minutes. If the second and third calls report a cache hit and you see the TTFT cut in half, you’ve just realized a real‑world saving.
