Probabilistic Architecture

Overview

This is the structural heart of the section. Almost everything else in it follows from one property.

The property

A traditional component is deterministic: same input, same output, every time. You test it by asserting the answer. When it’s wrong, it’s wrong the same way twice, and you can debug it.

A language model is probabilistic: the same input can produce different output on different runs. Usually not wildly different — but not identical, and occasionally wrong in a way it was right about a moment ago.

That single property is the whole topic.

Deterministic — short, and you can assert the answer:

graph LR
  D1["Request"] --> D2["Logic"] --> D3["Response"] --> D4["Assert:<br/>is it correct?"]

  classDef det fill:#7D9A4E,stroke:#5F7A38,stroke-width:2px,color:#ffffff
  class D1,D2,D3,D4 det

Probabilistic — the same request, with everything that has to surround it:

graph LR
  P1["Request"] --> P2["Input<br/>check"] --> P3["Model"] --> P4["Output<br/>check"] --> P5{"Confident<br/>enough?"}
  P5 -->|yes| P6["Response"] --> P8["Measure:<br/>correct often<br/>enough?"]
  P5 -->|no| P7["Fall back, or<br/>hand to a human"]

  classDef prob fill:#D9695B,stroke:#B4483A,stroke-width:2px,color:#ffffff
  classDef gate fill:#E8A33D,stroke:#C4832A,stroke-width:2px,color:#ffffff
  classDef exit fill:#7C7C7C,stroke:#5E5E5E,stroke-width:2px,color:#ffffff
  class P1,P3,P6 prob
  class P2,P4,P5 gate
  class P7,P8 exit

The same request path, both ways. The amber boxes are the checks and the decision — they are the design work, and they have no counterpart above.

Three consequences

Testing becomes measuring. You can’t assert the answer equals an expected string, because there isn’t one expected string. You measure quality across many examples and set a threshold. “Correct” becomes “correct often enough” — which means somebody has to decide what enough is.

That’s not a QA detail. It’s an architectural decision with a business owner. Is 90% accuracy acceptable when the other 10% reaches a human anyway? Probably. Is it acceptable when the action moves money? Different question entirely, and whoever owns that financial exposure belongs in the room for it.

Cost becomes variable and structural. A traditional service costs roughly the same per request. A model call costs in proportion to how much text goes in and comes out. Cost becomes a design parameter — how much context you include, which model handles which request, what you cache.

Where there’s a hard cost-per-interaction target, this isn’t an optimisation for later. It constrains the structure from the beginning, exactly as the chain in Why the job exists predicts a hard driver should.

Failure gets quiet. This is the dangerous one.

A deterministic system fails loudly — an exception, a 500, an alert fires. A probabilistic one degrades. It returns confident, plausible, wrong answers, and nothing in your monitoring notices, because nothing crashed.

The failure nobody noticed

In 2023 the company shipped a machine-learning feature for “smart routing” — automatically assigning jobs to technicians. It worked at launch. Over the following months it degraded. Eventually it was switched off, quietly, without an announcement.

There was no incident, no postmortem, no alert. Just a slow decline nobody was measuring.

Two things follow for the current initiative. First, the same failure mode is available again, and nothing in the existing monitoring would catch it. Second, the organisation remembers — which is why a proposal that sounds similar meets resistance that has nothing to do with its technical merits. Module 8 returns to that.

Designing for it

Three moves follow directly.

Decide the failure path first. What does a user see when the model is unavailable, or unsure? If you design the happy path first, the failure path becomes whatever’s left over.

Make confidence a control-flow decision. “Answer if confident, otherwise hand to a human” is a structural choice about where the boundary sits — and it needs a number attached, not a vibe.

Measure continuously, not once. Quality at launch tells you nothing about quality in six months. The model provider updates. The documentation drifts. Customer questions change. Without ongoing measurement you get the quiet failure above, again.

The cost: all three add latency, expense, and work that produces no visible feature. Skipping them is the cheaper option right up until it isn’t.

What doesn’t change

Worth stating plainly, because the discourse implies otherwise.

Business drivers still come first. Quality requirements still shape structure. Trade-offs are still trade-offs. Everything you know about distributed systems, failure modes, data consistency and security still applies — a system with a model in it is still a distributed system, with all the usual problems plus new ones.

The genuinely new parts are: output varies, cost scales with usage in a new way, failure is quiet, and the thing can act on its own. That’s a shorter list than the hype suggests, but each item has structural consequences.

Feeds the workbook

Your business-value brief must name at least one concrete consequence of unpredictable output. “The model might be wrong” isn’t enough — say what it would cost and what the system does about it.

Check your understanding

1. The 2023 feature in the case degraded quietly over months. What would have caught it, and where does it sit in the request path?
2. Who should decide what accuracy counts as “enough” for an action that moves money — and why isn’t it you?
3. Name a deterministic assumption in a system you’ve worked on that would break if one component became probabilistic.

---

→ If you can’t fully trust the output, what can you trust? Your own understanding of the domain — which is now worth more, not less. — Domain Authority