Cost Levers, in Order
Overview
Section titled “Overview”Module 4 introduced a lever order and Module 5 added bounds. This topic is the complete catalog, in the sequence that matters.
The ordering principle is simple and it is the whole discipline: levers that do not cost quality go first, and levers that trade quality go last. Working out of order means spending accuracy you did not need to spend.
Two kinds of lever
Section titled “Two kinds of lever”graph TD
subgraph F["Free wins — no quality cost"]
A["1 · Prompt caching<br/><i>largest single lever;<br/>stays on permanently</i>"]
B["2 · Input hygiene<br/><i>send what the task needs</i>"]
C["3 · Loop hygiene<br/><i>bounds, narrow accessors</i>"]
D["4 · Output shaping<br/><i>specify the shape</i>"]
E["5 · Batch<br/><i>where nobody waits</i>"]
end
subgraph T["Trade-offs — quality is the currency"]
G["6 · Reasoning effort"]
H["7 · Task budgets"]
I["8 · Model choice"]
J["9 · Multi-model"]
end
E ==> G
classDef free fill:#7D9A4E,stroke:#5F7A38,stroke-width:2px,color:#ffffff
classDef trade fill:#D9695B,stroke:#B4483A,stroke-width:2px,color:#ffffff
class A,B,C,D,E free
class G,H,I,J trade
Nine levers in two classes. Everything olive is safe to apply without an evaluation; everything terracotta requires one, because you cannot tell a saving from a regression without it.
The free wins
Section titled “The free wins”Prompt caching, first and permanently. The previous topic showed why: in a loop it is the difference between affordable and not. Two practical notes. It requires prefix stability, so anything volatile belongs after the last breakpoint. And it must be verified from usage data rather than from code review — on a warm loop, cache reads should dominate fresh input, and if they do not, something is invalidating the prefix and no amount of cache_control in the source will tell you.
Input hygiene — progressive disclosure. Send what the task needs and let the system fetch the rest. A large reference document inlined into every request moves behind a tool. Prose in the system prompt restating what tool schemas already say gets deleted. Images and files get downscaled to what the task actually requires. Broad data-dump tools become narrow accessors with limits and field selection.
The caveat that governs this whole lever: a smaller prefix is not automatically a cheaper task. Deferring context means the system may spend discovery turns fetching what it previously read inline, and those turns are subject to the squared-growth arithmetic. This one needs validating rather than assuming.
A specific sub-lever worth naming: prompts written for an older model make a newer one over-work. On a support-desk evaluation, prompts carried forward unchanged cost 36% more per ticket on a newer model with no accuracy gain; audited and updated, the same prompts ran 14% cheaper and more accurate — 97% of tickets handled against 92%. That is a free win hiding in text nobody has reread.
Loop hygiene. The bounds from Module 5, plus the correction from the previous topic: pruning manages the window, not the bill.
Output shaping. Specify the exact output shape in the prompt, ideally with an example. A sentinel the system emits when it cannot proceed lets it stop rather than spending tokens explaining itself at length.
Batch, where nobody is waiting. Roughly half off every token including cache reads and writes, so the discounts stack.
The trade-offs
Section titled “The trade-offs”From here, every lever spends quality, which is why they come after an evaluation exists.
Reasoning effort scales how much the model thinks without changing which model it is. Sweep this before touching the model — and the expected shape varies by workload. On research and knowledge work the curve is often nearly flat: published sweeps found a middle setting matching the default’s accuracy at 70–85% of its cost. On long-horizon coding it is a genuine trade: a step down gave up about two points of pass rate for half the cost. On work at the reasoning ceiling there was no free cut at all.
Task budgets — the advisory kind from Module 5 — save money because the system paces itself. Measured on coding, a generous budget gave up about 2.7 points for an 18% saving; the tightest allowed gave up 4.4 points for 47%. Set it from the loop’s observed high-percentile usage, then tighten.
A variant worth knowing because it is measured and counter-intuitive: run everything cheap and re-run only the failures expensively. On a coding workload that produced roughly the same pass rate for about half the total cost, counting the wasted cheap attempts. It requires a usable failure signal — a test, a validator, a checker — which is exactly what Section 3 is about building.
Model choice, last and deliberately, because it constrains the intelligence ceiling. Price candidates in cost per completed task on your own traffic, including the stronger model at reduced effort, and compare on the hardest tenth rather than the median.
Multi-model architectures pay in narrow, measured shapes — and Module 5’s arithmetic already showed how serial composition multiplies reliability downward. Published comparisons found both the advisor and orchestrator patterns sitting within noise of a single strong model at moderate effort on several workloads, and reversing on others. Treat them as architecture changes requiring validation, not as optimisations.
One lever per change, measured
Section titled “One lever per change, measured”The process discipline, briefly: apply one lever at a time, measure cost and quality together against the previous kept configuration, and revert anything that saves money while giving back accuracy. A lever that saves 15% and costs three points of correctness is not an optimisation — it is a decision somebody with a business stake should make explicitly.
Expect most levers not to apply. On a published worked example most did not earn a place — tool schemas too small to defer, loops too short for window management, no numeric work to offload — and the ones that came closest each gave back a correct answer.
Feeds the workbook
Section titled “Feeds the workbook”The rubric asks for the levers that get you to target. Rank them by ceiling, apply them in class order, and mark every trade-off as blocked until an evaluation exists — that honesty is what makes the model defensible.
→ The levers are known. Someone still has to own the number, notice when it moves, and stop it. — Cost governance