Your AI coding session gets pricier with every message. Here's why.

Type “rename this variable” as the 8th message of a Claude Code session. Now type it again as the 80th. Same words, same answer, and with my sample numbers the second one costs about three times as much. Turn prompt caching off and it’s six.

Nothing’s broken. A long session just walks into a quadratic curve twice: once inside the model, where every token looks at every earlier token, and once on your bill, where every turn ships the whole conversation again. Caching makes the repeat cheaper. It doesn’t change the shape. There’s a simulator at the end if you’d rather poke than read.

The model has no memory

Claude Code’s docs put it bluntly: “The model doesn’t remember anything between requests, so Claude Code re-sends the full context.” And one message from you is rarely one request. Every tool call sends its result back as a new request carrying everything before it. A long session is a stack of requests, each a little fatter than the last.

What attention actually does

A token is a chunk of text, roughly ¾ of an English word. Inside the model it becomes a vector of a few thousand numbers, and attention is where tokens look at each other. It’s how “it” figures out which noun it means.

Each token gets three vectors. The query is what it’s looking for, the key is what it offers, and the value is what it hands over if someone picks it. A token compares its query with every key (multiply element by element, add it up), softmax turns those scores into weights that sum to 1, and the output is the weighted mix of values. The 2017 paper fits it on one line:

The √d_k is there so the scores don’t get big enough to break softmax.

Everyone checks everyone, so the work is n²

Q·Kᵀ is every query against every key: n tokens, an n × n grid. The paper lists self-attention at O(n²·d) per layer, which means double the tokens and you get four times the work. Language models add one rule, that a token can only look backwards. That trims the grid to a triangle of about n²/2 cells. Still a square, just a lazier one.

At 1,000 tokens the triangle has about half a million cells. At 100,000, which a coding session hits after reading a few files, about five billion. At 1 million, the full window on current Claude models, about 500 billion, per head, per layer. FlashAttention fixed the memory problem by never storing the whole grid, but it still computes every cell, so the time stays quadratic.

The KV cache helps, but not how you’d think

A reply comes out one token at a time. Old tokens’ keys and values don’t change when a new one arrives, so the server keeps them in GPU memory instead of rerunning everything. kipply’s write-up says the point is “to avoid recalculations of those vectors every time we sample a token.”

Here’s the part plenty of explainers get wrong: the cache doesn’t make generation linear. Each new token still does one dot product per earlier token, so a whole reply is still about n²/2. What the cache saves you is the rerun, which would push it toward cubic. Sebastian Raschka puts it as decoding work per token that “grows linearly with the retained prefix.”

None of this shows up on your bill as such. Anthropic says a 900k-token request “is billed at the same per-token rate as a 9k-token request.” So if the model’s own n² isn’t what you pay for, what is?

Your bill is quadratic too, for a dumber reason

Say a session starts with S tokens (system prompt, tools, CLAUDE.md) and each turn adds about a. Turn k sends roughly S + k·a, so K turns add up to about S·K + a·K²/2. That second term is the triangle again: every turn is a row, and every row re-sends everything above it.

Prompt caching lowers the slope, not the shape

If the start of a request exactly matches something processed recently, the API skips the recompute and bills that part much cheaper. Anthropic doesn’t say what it stores. The open-source vLLM server does “cache the kv-cache blocks of processed requests”, so my guess is it’s the KV cache from Demo 2, kept around between requests.

Prices from Anthropic’s pricing page (checked October 2026):

A cache read costs 0.1× the input price (0.05× on Opus 5.5 and Sonnet 5.5) and resets the timer for free. Claude Code sets all this up for you. A 20× discount sounds like it solves everything, but the cached part is exactly the part that grows. You still re-read the whole history every turn, just at a fraction of the price, and a fraction of a quadratic is still a quadratic.

Watch the timer, too. It runs from the start of a request, and Claude Code’s docs say it’s an hour on a subscription but five minutes on an API key or once you’re on usage credits. Grab a coffee on an API key and your next message rewrites the whole context at the write price.

With the defaults (Sonnet 5.5, a 20k-token start, 3k tokens added per message, four requests per message) message 8 costs about $0.05 and message 80 about $0.16. The whole 80-message session comes to about $8.10, or about $112 with no caching at all. Cache reads are around two-thirds of the cached bill. That’s the re-sending, on sale.

What actually moves the curve

Each of these matches Anthropic’s docs, and you can try them all in the simulator.

• /clear resets the triangle. A /clear at message 41 takes the default session from about $8.10 to about $5.70. The system prompt and tools stay cached.
• /compact is a big request itself. It reads the whole conversation to write the summary, so it saves about what /clear does but costs one large read first. Worth it when you need the continuity.
• Long breaks hurt most late in a session. A 20-minute break before message 60 on the 5-minute cache takes that one message from about $0.13 to about $0.72. At message 5 you’d barely notice.
• Switching models throws the cache away. So does connecting an MCP server whose tools load up front, and on most models so does changing effort mid-session.
• Subagents get their own context. Their noisy output stays out of your main conversation, so it isn’t re-sent on every later message.

The fine print

The simulator spreads tokens evenly and real sessions don’t, so run /usage for your actual numbers. On Pro or Max you hit usage limits rather than a dollar figure, but the same curve decides how fast. And the attention maths is textbook, not Claude: Anthropic doesn’t publish its architecture, so tricks like grouped KV heads can change the constants, though not the n² shape of standard attention.

What I’d actually do

Treat a session like a branch, not a home. One task per session, /clear when the task changes. Keep CLAUDE.md short, since it rides along on every request (Anthropic suggests under 200 lines). Pick the model and effort at the start and leave them alone. If you’re on an API key and you wander off mid-task, set promptCacheTtl to 1h or /compact before you go. And when a test run or log dump is going to be huge, let a subagent read it and hand back the summary.

None of that changes the shape. It just keeps you on the cheap end of it.

Sources

1. Claude Code docs, managing costs: docs.claude.com/en/docs/claude-code/costs
2. Vaswani et al., Attention Is All You Need (2017): arxiv.org/abs/1706.03762
3. Dao et al., FlashAttention (2022): arxiv.org/abs/2205.14135
4. kipply, Transformer Inference Arithmetic: kipp.ly/transformer-inference-arithmetic
5. Sebastian Raschka, Understanding and Coding the KV Cache in LLMs: magazine.sebastianraschka.com
6. Anthropic, context windows: docs.claude.com/en/docs/build-with-claude/context-windows
7. Anthropic, prompt caching: docs.claude.com/en/docs/build-with-claude/prompt-caching
8. Anthropic, pricing: docs.claude.com/en/docs/about-claude/pricing
9. vLLM, automatic prefix caching: docs.vllm.ai

All demo values are computed live in your browser from the formulas above. Prices are from Anthropic’s pricing page as of October 2026.