Observability for Agents

Overview

Last Tuesday the system issued a credit it should not have. A customer has noticed, the support lead wants an explanation, and the compliance owner wants to know whether it can happen again.

You have today’s logs and today’s system. This topic is about whether that is enough — and about the one thing people record that turns out not to help.

You cannot reproduce it

Conventional debugging has a step everyone relies on: run it again with the same inputs. That step is gone.

The same request to a bounded loop can take a different number of steps, call different tools in a different order, and reach a different conclusion. Re-running Tuesday’s interaction does not reproduce Tuesday’s fault; it generates a fresh interaction that happens to start the same way.

graph LR
  A["Deterministic<br/>component"] --> B["Re-run the inputs"] --> C["Fault reproduces<br/><i>the record is<br/>a convenience</i>"]
  D["Agent"] --> E["Re-run the inputs"] --> F["A different path<br/><i>the record is the<br/>only evidence</i>"]

  classDef det fill:#7D9A4E,stroke:#5F7A38,stroke-width:2px,color:#ffffff
  classDef prob fill:#D9695B,stroke:#B4483A,stroke-width:2px,color:#ffffff
  class A,B,C det
  class D,E,F prob

The same investigative step, two outcomes. On the lower path, recording is not a debugging aid — it is the primary artifact, and anything not captured at the time is gone.

Which inverts the usual priority. In a deterministic system, logging is a convenience that saves you a reproduction. Here, recording is the product — and anything you did not capture at the time is permanently unavailable.

What the record must contain

To answer why did it issue that credit, a trace needs all of the following. The test for each item is that omitting it leaves a question you cannot answer.

The inputs, including every retrieved chunk with its provenance and version. Without this you cannot distinguish a reasoning failure from a bad source — and after Module 4, you know a stale document is the likelier culprit.
Every tool call, with its exact arguments and its result. Not a summary. The arguments are the decision.
The plan, and any revision to it. If the pattern produced one, it is the closest thing to an intent you will ever have.
The classification decision — which blast-radius tier applied, whether confirmation was required, and if so who gave it and when.
Which bound terminated the loop, from the four in Section 1.
Token and cost attribution per step, which Module 2’s p95 ceiling requires and which no aggregate figure can reconstruct.
The model and configuration version, because the system that ran on Tuesday is not the system running today.

Taken together, that list reconstructs a decision without reference to the model’s internal state — which matters, for the reason below.

The trap: auditing the reasoning

The intuitive thing to record is the model’s reasoning. It is also the thing not to depend on, for two independent reasons.

It may not be available. Reasoning output is frequently summarised, omitted by default, or returned in a form you cannot read. A trace design that assumes a readable account of the model’s thinking will quietly have an empty field in it, and you will discover this during an incident.

It is not a reliable account even when present. A generated explanation of a decision is another generated output — fluent, plausible, and not necessarily a faithful description of what actually drove the behaviour. Treating it as a causal account is exactly the “it looks right” failure Module 1 warned about, applied to forensics.

So record the observable facts: what went in, what was called, with what arguments, what came back, which gates fired, what was decided. Those are checkable. Capture reasoning output where it is available, as useful colour — and never build an audit obligation on it.

Cost attribution, and the store itself

Per-interaction cost belongs in the trace because Module 2 made it a requirement with a p95, and if Module 4’s cascade is in play then per-tier attribution is the only way to see a cheap tier quietly degrading while the blended number stays green.

Two things about the trace store, both easy to defer and expensive to retrofit. Its retention window is set by the audit cycle, not by engineering convenience — compliance decides, and the SOC 2 period is the floor. And traces contain personal data: the conversation, the retrieved tickets, the customer identifiers. The store therefore inherits every obligation from Module 4’s data strategy, including the erasure path. A right-to-erasure request that cleans the database and the index and leaves the traces untouched has not been satisfied.

What it costs

Full-fidelity traces of every interaction are a storage and pipeline cost, and at the volume of a support platform they are not trivial. Sampling is the usual answer, and it is the wrong answer for the action path: you cannot sample the credit you will be asked about. The defensible split is everything for consequential actions, sampled for read-only answers — which keeps the cost proportionate while preserving the evidence that matters.

The other cost is latency: writing a rich trace synchronously is work inside the request path. Write it asynchronously, and accept that a crash can lose the tail of a trace — a trade worth making explicit rather than discovering.

The record for one credit

A credit was issued on Tuesday. What exists:

Captured	Why it matters here
The customer’s messages, and every retrieved chunk with source and version	Lets you check whether a stale documentation page suggested an entitlement that no longer exists
The plan as stated: “issue a $40 credit against ticket 8812, because the visit was missed”	The intent, in a form a human read and approved
`check_entitlement` call, arguments, result	Shows whether the system verified eligibility or assumed it
`issue_credit` call with amount and idempotency key	Distinguishes one credit from a retried one
The tier decision: ask first, confirmed by a named agent at a timestamp	Answers Dev Raghunathan’s question — a human approved this
Terminating bound: none; the loop completed in 5 steps	Rules out a truncation
Cost: per-step tokens and total	Confirms the interaction sat inside the $2.50 p95
Model and config version	The system that ran then, not now

With that, a support engineer answers the customer in minutes and Lena Fischer can see that the control fired as designed. Without the entitlement call and its result, the investigation ends at “the model decided to”, which is not an answer anyone can act on.

Retention: set by the SOC 2 audit cycle, and the traces are in the erasure path alongside the primary store and the vector index.

Sampling: every action-path interaction is traced in full. Read-only answers are sampled, because nobody will ever ask which documentation page was cited in a question that was answered correctly and forgotten.

Feeds the workbook

This is the observability plan. The rubric asks that it would let you reconstruct a specific bad decision after the fact — so write it against that test: pick the credit, and check your plan captures every field the investigation needs.

→ One more exit remains, and it is the one the support team will judge the whole system by. — The human handoff