September 2, 2026
From 34% to 90%: caching an agent’s prompt
One line that changed on every step was costing us two thirds of our cache. The fix was not to cache harder. It was to move the line.
An agent that works on a canvas has to be told what's on the canvas. We put that in the system prompt, which is the obvious place to put it, and it quietly cost us most of the benefit of prompt caching.
The rule, which is not obvious
A provider serves the longest matching prefix of your whole request from cache. Not the parts that match: the prefix. So a single line that changes on every step stops everything behind it caching, wherever in the system prompt it happens to sit, including a transcript that's otherwise identical to last time.
Canvas state changes on every single step. Sitting it in the system prompt meant our prefix diverged immediately, every time, forever.
What we did
We split the prompt by how often things change instead of by what they're about:
- The stable half holds the core instructions, the surface reference, the project name, the brief and memory. It's constant for a whole run, so it makes a good cacheable prefix.
- The volatile half holds the plan, the current part, canvas state and the step instruction. It's delivered as a trailing message after the transcript, not inside the system prompt.
What it was worth
Measured across 24 scenarios and 63 director calls:
| Canvas state placement | Input tokens | Cached |
|---|---|---|
| Inside the system prompt | 343,987 | 118,114 (34%) |
| After the transcript | 351,437 | 316,874 (90%) |
Slightly more tokens in total, and nearly three times as many of them cached. The extra tokens are the framing that moving the block needed, and the cache pays for them many times over.
The rule we wrote down afterwards: never move canvas state back into the system prompt. It reads as tidier and it costs two thirds of the cache.
Why this probably applies to you too
Any agent with a stable brief and a volatile world state has this shape. The instinct is to assemble one coherent prompt that describes everything nicely. Prefix caching punishes exactly that. Order your request by how often each part changes, not by how related the parts are.