How does prompt caching reduce AI costs?
Don't pay full price to re-read the same pages every time.
Picture it first
Every morning your teacher reads the same 10 class rules before the lesson. That wastes time. So she prints them once, sticks them on the wall, and just points: 'Rules as usual!' Only the new part — today's lesson — is read out loud. But if she changes even one word in the rules, she has to print a new poster.
Now the same idea, in simple QA words
Many AI requests start with the same long text — the same instructions, the same tool list, the same document. Prompt caching lets the AI provider save that repeated beginning. Next time, it is re-used at a much cheaper price and faster. The catch: the saved part must be exactly identical. Change even one character at the start (like adding today's time) and the saving is lost. Rule: put fixed text first, changing text last.
How the story connects to the real thing
Real-world example
A test-case generator sends a 12,000-token system prompt (standards, templates, examples) + a 500-token user story. Without caching, every call pays for 12,500 input tokens. With caching, the 12,000-token prefix is written once and read cheaply on later calls within the cache lifetime. A developer later adds a timestamp at the top of the system prompt 'for debugging' — the cache hit rate drops to zero and costs jump back up. The tester catches it on the cost dashboard.
Step by step, in plain words
- Put the fixed parts (instructions, tools, documents) at the start.
- Put the changing part (user question) at the end.
- First request: the fixed part is saved.
- Next requests: the saved part is re-used cheaply.
- Check the usage report to confirm the cache was used.
- Keep the fixed part unchanged.
How a tester tests it
- Assert cache hits in the API usage fields (e.g. cache_read_input_tokens / cached_tokens) for repeat requests.
- Regression-check that nobody puts dynamic data (timestamps, user names, random IDs) in the prefix.
- Verify answers are identical in quality with caching on vs off (caching must not change behaviour).
- Track cache-hit rate and cost per request as monitored metrics.
Try it yourself
1) Look at one of your prompts and split it into 'never changes' and 'changes every time'. 2) Reorder it so all stable parts come first.
Interview angle
“How is prompt caching different from caching the final answer?”
Prompt caching reuses the processed prefix; the model still generates a fresh answer for each new question. Response caching returns a stored answer for a repeated question without calling the model. Semantic caching goes further and reuses answers for similar questions — which is cheaper but risks serving stale or wrong answers, so it needs its own tests.