The Autonomy Spectrum

Overview

“Agent” has become a word people use for anything with a model in it, which makes it useless for deciding anything. This topic replaces it with a spectrum.

The spectrum matters because the choice is not binary. There are several positions between one model call and an autonomous system, and the right answer for most problems is somewhere in the middle.

The positions

graph LR
  A["<b>Single call</b><br/>one request,<br/>one response"] --> B["<b>Chain</b><br/>fixed sequence<br/>you wrote"]
  B --> C["<b>Router</b><br/>model picks<br/>a branch"]
  C --> D["<b>Tool loop</b><br/>model chooses tools,<br/>iterates until done"]
  D --> E["<b>Autonomous agent</b><br/>model plans, re-plans,<br/>decides when finished"]

  classDef pred fill:#7D9A4E,stroke:#5F7A38,stroke-width:2px,color:#ffffff
  classDef mid fill:#E8A33D,stroke:#C4832A,stroke-width:2px,color:#ffffff
  classDef loose fill:#D9695B,stroke:#B4483A,stroke-width:2px,color:#ffffff
  class A,B pred
  class C,D mid
  class E loose

Five positions. Multi-agent sits further right still and gets its own section. Colour is predictability: olive is control flow you wrote, terracotta is control flow the model decides.

Single call. One request, one response. Entirely predictable in structure, if not in content.

Chain. A fixed sequence you wrote: retrieve, then generate. Retrieval-augmented answering is a chain, not an agent — a distinction worth insisting on, because calling it an agent invites agent-shaped costs for pipeline-shaped work.

Router. The model picks a branch, your code executes it. One decision is delegated and the rest of the flow remains yours. This position is underused and often sufficient.

Tool loop. The model chooses which tools to call, sees the results, and decides whether to continue. Now the number of steps is not known in advance.

Autonomous agent. The model plans, revises the plan when reality disagrees, and decides when it is finished. The control flow has become an output of the system rather than a property of it.

What each step to the right transfers

One sentence carries the whole topic:

Every step right moves control of the program’s flow from code you wrote to a decision the model makes at runtime.

That is the thing being traded. Not intelligence, not sophistication — predictability. And four costs arrive with it, each of which has shown up in earlier modules in a milder form.

Non-determinism moves into control flow. Module 1 established that output varies. Here the path varies: the same request may take three steps or eleven, call different tools in a different order, and arrive by a route nobody anticipated. Testing a value is hard; testing a path is harder.

Latency compounds. Each iteration is a round trip, and the round trips are sequential by construction, because step four depends on what step three returned. A system with a p95 answer budget of a few seconds can take a minute once it is allowed to iterate.

Cost variance becomes unbounded. A chain costs roughly the same every time. A loop costs whatever it decides to cost, and the p95 is not a modest multiple of the median — it is whatever the worst reasonable path happens to be. Module 2 asked for a p95 cost ceiling; this is the position on the spectrum where that number stops being theoretical.

Debuggability degrades. When a chain misbehaves you inspect a known step. When an agent misbehaves you reconstruct a path that will not recur if you run it again. This is why observability is a Section 2 topic rather than an afterthought.

The honest default

Module 1 put it as the leftmost thing that works, and the reasoning is stronger here than it sounds.

The costs above are not fixed overheads you pay once; they scale with how much latitude you grant. A router costs slightly more than a chain. An autonomous agent costs considerably more than a tool loop and is considerably harder to explain to a compliance owner. If two positions both solve the problem, the left one is better on every axis except elegance.

The common mistake is choosing a position for the system rather than for each path through it. A system can be a chain for one kind of request and a bounded tool loop for another, and usually should be — which is also an argument for the separate deployables Module 3 settled on.

Two paths, two positions

The platform has two jobs, and they do not belong at the same point on the spectrum.

Answering a documented question is retrieve, then generate, then check. That is a chain — a fixed sequence, two or three steps, no iteration. It handles the 38% of tickets the documentation already answers, and giving it latitude to iterate would add latency and cost variance to a path whose shape is already known. Module 4’s decision framework reached the same conclusion from the retrieval side.

Taking an action — rescheduling a job, issuing a credit, resetting a device — is different. The system must establish which job, check whether the change is permitted, make it, and verify it took effect. The number of steps depends on what it finds: a job that does not exist, a customer with no entitlement, a device already offline. That is a bounded tool loop, and the bound is the subject of two topics from now.

What neither path is: an autonomous agent that decides its own objectives. There is no task here whose goal is unknown at the start, and granting that latitude would buy nothing while making Ivan Petrov’s 2023 objection considerably harder to answer.

The architectural consequence is that the retrieval path and the action path have different failure modes, different cost profiles and different oversight requirements — which is the clearest possible justification for the Module 3 decision to make them separate services rather than one.

Feeds the workbook

Your agent design must state which position each path occupies and why. The rubric asks for the agent-versus-pipeline decision to be argued; naming the position for each path separately is how you argue it without overclaiming.

→ The spectrum gives positions. It does not say which one you are entitled to. — Should this be an agent?