Hybrid Architectures

Overview

Five topics of machinery for handling uncertainty. This one asks the question underneath all of them: how much uncertainty do you need to handle?

The claim of this topic is deliberately provocative and I think it is right:

The strongest AI architectures contain the least AI.

Not because models are bad, but because a model is the most expensive, least predictable and hardest-to-verify component available, and it should therefore be used for exactly the parts of a problem that need it.

The asymmetry that makes hybrids work

Here is the structural insight.

For a large class of problems, checking an answer is far cheaper and more reliable than producing one. Is this date in the future? Does this job belong to this customer? Is this amount within the limit? Does this reference exist? Does this JSON match the schema? Every one of those is arithmetic or a lookup — deterministic, instant, free, and certain.

That asymmetry is the whole design pattern. Generate with the probabilistic component, then verify deterministically, and reject or retry when verification fails. You get the model’s flexibility on the part that needs flexibility, and a guarantee on the part that can be guaranteed.

graph LR
  A["Request"] --> B["Generate<br/><i>probabilistic</i>"]
  B --> C{"Verify<br/><i>deterministic</i>"}
  C -->|"passes"| D["Release"]
  C -->|"fails"| E["Reject, retry,<br/>or degrade"]
  E --> B

  classDef prob fill:#D9695B,stroke:#B4483A,stroke-width:2px,color:#ffffff
  classDef det fill:#7D9A4E,stroke:#5F7A38,stroke-width:2px,color:#ffffff
  classDef plain fill:#5B8FB9,stroke:#4A7A9E,stroke-width:2px,color:#ffffff
  class B prob
  class C,D det
  class A,E plain

Generate-then-verify. The olive gate turns a probabilistic output into a guaranteed property — but only for the properties you can actually check.

Where to put the deterministic parts

Four placements, each with a different purpose:

Before generation — constrain the input. Resolve ambiguity deterministically first. If the customer’s question mentions a job number, look it up and supply the facts rather than letting the model infer them. Every fact you establish before generation is a fact the model cannot get wrong.

Instead of generation — rules where rules work. A policy with twelve clear conditions is a decision table, not a judgement. It is cheaper, instant, auditable, and testable, and routing it through a model buys nothing but variance. This is the placement teams skip because the rules feel unglamorous.

After generation — verify. The pattern above. Schema validation, arithmetic checks, existence checks, policy checks.

Around generation — constrain the output space. Where the answer must be one of a known set, enforce that structurally rather than hoping. Constraining the shape of what can come out is far more reliable than detecting bad shapes afterwards.

The umbrella term for combining learned and rule-based components is neuro-symbolic, and it is worth knowing in a literature search. For practical purposes the four placements above are the whole technique.

What you cannot verify

Being honest about the limit, because generate-then-verify is sometimes oversold.

Verification only works for properties with a decidable check. You can verify that an answer cites a retrieved document; you cannot verify that it is a good answer. You can verify that a credit is within the limit; you cannot verify that it was deserved. The parts of the problem that genuinely require judgement remain probabilistic, and that residue is what the previous five topics are for.

So the design question is not “can I verify everything?” but “what fraction of the risk can I move into the deterministic part?” — and the answer is usually a much larger fraction than the first design assumed.

The cost

Rules require maintenance by people who understand the domain, and a decision table with forty branches that nobody has reviewed in two years is its own antipattern. Verification code is code, with its own bugs, and a buggy verifier that passes bad output is worse than none because it manufactures confidence. Constraining output can make answers stilted where flexibility was the point.

And there is an organisational cost worth naming: hybrid designs are less exciting. “We replaced the model call with a lookup” is the right answer and it does not present well. That is a real force, and recognising it is part of the job.

Where the AI gets removed

Auditing the Modules 4–6 design for places a deterministic component should replace a probabilistic one — which is one of this module’s rubric items.

Before generation. When a question references a job, the job is looked up and its facts supplied: status, scheduled slot, site, assigned technician. Previously the model would have inferred some of this from retrieved text. Facts supplied deterministically cannot be hallucinated.

Instead of generation — the clearest win. Entitlement checking. Whether a customer is entitled to a credit is a policy with a small number of conditions: contract tier, incident type, whether a visit was missed, whether a credit was already issued for this ticket. That is a decision table. Module 5 had check_entitlement as a tool the agent calls and reasons about; the stronger design has the table return a decision, and the agent explain it. The model keeps the part it is good at — phrasing the explanation — and loses the part it should never have had.

After generation. Every answer must cite a document retrieved in this interaction; the citation is checked structurally. Every credit is checked against the limit in the action layer, which Module 5 already required. Every reschedule is verified by reading the job back, which the retry topic required.

Around generation. The plan the action path produces is constrained to a known set of action types with typed arguments, rather than free text later parsed. A plan that cannot be expressed in that shape is rejected before anyone sees it.

What stays probabilistic, legitimately: understanding what a customer is actually asking, in their own words, with typos and missing context; selecting which documentation is relevant; and phrasing an answer a non-expert can act on. That is the part worth paying for, and it is a much smaller surface than the original design implied.

The number worth noticing. Of the ways this system could produce a wrong outcome, the majority are now caught by checks that cannot themselves be wrong about the thing they check. Module 6’s threat model and Module 5’s action boundary were already most of the way there; this topic names the pattern and finds the entitlement table, which nobody had spotted.

Feeds the workbook

The rubric requires at least one place where a deterministic component should replace a probabilistic one. Audit your own design with the four placements as a checklist — most designs have at least one decision table hiding inside a model call.

→ That closes the engineering section. Next: the failure patterns that recur, starting with the one that compounds. — Hallucination amplification