Why Loops Cost What They Do

Overview

Module 5 bounded the agent loop for predictability and safety. This topic shows that the same bound is one of the largest cost levers in the system, for a reason that is arithmetic rather than judgement.

If you take one piece of arithmetic from this module, take this one.

The resending problem

The API is stateless. Every turn of a loop resends the entire conversation so far — the system prompt, the tool definitions, and every previous turn with its results.

So the first turn is sent once. The second turn resends turn one. The third resends turns one and two. By turn twelve, turn one has been transmitted twelve times.

Total input volume is therefore not proportional to turn count. It is proportional to the sum of turn counts — roughly the square of the number of turns.

graph LR
  A["Turn 1<br/>sent 12×"] --> B["Turn 6<br/>sent 7×"]
  B --> C["Turn 12<br/>sent 1×"]
  C --> D["<b>78 turn-equivalents</b><br/>for a 12-step loop,<br/>not 12"]

  classDef early fill:#D9695B,stroke:#B4483A,stroke-width:2px,color:#ffffff
  classDef mid fill:#E8A33D,stroke:#C4832A,stroke-width:2px,color:#ffffff
  classDef late fill:#7D9A4E,stroke:#5F7A38,stroke-width:2px,color:#ffffff
  classDef tot fill:#5B8FB9,stroke:#4A7A9E,stroke-width:2px,color:#ffffff
  class A early
  class B mid
  class C late
  class D tot

Early turns are the expensive ones because they are resent most. A 12-step loop transmits 78 turn-equivalents of input — the sum of 1 through 12 — which is why doubling a step cap roughly quadruples input cost.

Three things follow immediately.

A step cap is a cost control. Module 5 derived a cap of 12 from observed task shapes and safety. The same number, read through this arithmetic, caps input volume at 78 turn-equivalents. Raising it to 24 would not double the cost — it would take the total to 300, roughly four times as much. Any conversation about loosening a step cap is a conversation about cost, whether or not anyone says so.

Caching is not optional in a loop; it is the thing that makes loops affordable. Caching does not stop the resending — it reprices it. Of the 78 turn-equivalents above, 66 are repeats of material already sent. Repriced at roughly a tenth, those 66 cost about 6.6, bringing the total to roughly 18.6 instead of 78 — about four times cheaper. Published measurements on real agent loops land in the same region: cost reductions of roughly two-and-a-half to nearly four times at hit rates between 81% and 90%, with one small triage agent’s bill falling by 83% from caching alone.

Those are other people’s workloads and should be treated as sizing guidance rather than promises. The mechanism, though, is arithmetic and will hold for yours.

Anything that breaks the cache mid-loop is expensive in proportion to where it happens. A cache miss at turn ten resends everything at full price and rewrites it. Module 4 listed the invalidators; the ones that bite inside a loop are changing the model mid-conversation, altering the system prompt, changing the reasoning-effort setting between turns, modifying the tool set, and anything that rewrites earlier history.

The trap: pruning is not a saving

This is where a mechanism from Module 5 needs a correction, and it is a correction worth making explicitly.

Module 5 presented pruning — clearing stale tool results from the transcript — as a way to manage the context budget. It is. But it is not a cost lever, and in published measurement it cost more than it saved.

The reason is the arithmetic above. Pruning rewrites the conversation, and a rewritten conversation no longer matches the cached prefix. Every pruning pass is therefore a cache miss, paid at full price for everything after the edit. Prune to keep a long loop inside its window; do not prune to save money, and set the trigger high enough that passes stay infrequent rather than running every turn.

Summarisation behaves differently: it needs a long enough session to trigger at all, and where it does fire on a genuinely long run it can take a further substantial bite out of the bill — one measured run saw 38%. But it carries the fidelity cost Module 5 described, so it is a window mechanism with a cost side-effect rather than a cost lever with a quality side-effect.

Output caps are a backstop, not a knob

One more correction with money attached.

A maximum-output setting is enforced and the model cannot see it. When it fires, the response is cut off mid-thought — and a truncated attempt is a failed attempt that still billed. Measured on coding workloads, a tight cap ended 15% of one model’s attempts and a third of another’s, and none of those truncated attempts solved the problem. The capped runs spent less per attempt and bought proportionally fewer completions, so cost per completed task did not improve at all.

Which is the unit from the previous topic doing its work. Set the cap as a backstop well above what the task needs, treat a truncation as a failure rather than something to retry at the same ceiling, and shape output length through the prompt and the reasoning-effort setting instead.

This is also the clearest illustration of Module 5’s advisory-versus-enforced distinction. An advisory budget lets the loop land gracefully; an enforced cap guarantees you pay for work you then throw away.

What the step cap actually buys

The action path is capped at 12 tool calls, with a p95 cost ceiling of $2.50 and a monthly inference ceiling of $8,000.

The cap as a cost control. At 12 steps the loop transmits 78 turn-equivalents of input. Uncapped — or capped at a comfortable-sounding 24 — it would transmit 300. The step cap derived in Module 5 for safety is simultaneously holding input volume to roughly a quarter of what a looser cap would allow, and nobody framed it that way at the time.

Caching, with the confirmation gate complication. Repricing the 66 repeated turn-equivalents at roughly a tenth brings the effective total to about 18.6 — the difference between a path that fits inside $2.50 at p95 and one that does not. But the previous topic established that every credit waits on a human confirmation from one of the 45 agents across three time zones. A pause of minutes expires a short-lived cache entry, and the resumed loop then pays full price for its entire history and rewrites it, at exactly the point where the history is longest.

So the longer cache duration is not a micro-optimisation on the action path. It is what stops the oversight model from being the single largest line item in it.

What this reframes. Two decisions made for other reasons — Module 5’s step cap, and Module 5’s confirmation gate — turn out to be among the largest cost factors in the system. The cap helps; the gate hurts, and the mitigation is a cache-duration choice driven by Grace Adeyemi’s coverage schedule.

The arithmetic here is illustrative: it assumes turns of roughly equal size and a cache read at about a tenth of fresh input. Verify both against current pricing and your own measured turn sizes before using the figures.

Feeds the workbook

Your unit-economics model should show the loop arithmetic for the action path, not just a per-call figure. The rubric wants the levers that get you to target — and on an agentic path the step cap and the cache duration are two of the largest.

→ The arithmetic says where the money is. Here are the levers, in the order that does not cost you quality. — Cost levers, in order