Fallback and Degradation

Overview

Retries handle the case where trying again helps. This topic handles the case where it does not.

Module 3 introduced the degradation ladder as a quality tactic. Module 5 gave it entry conditions. This topic specifies it — which means answering a question most designs leave open: what does the user actually see?

Design the failure path first

Module 1 said it and it is worth making operational.

If you design the happy path first, the failure path becomes whatever is left over — an error message somebody wrote in a hurry, a spinner that never resolves, a response that silently omits what it could not do. The failure path gets the attention nobody had left.

Inverting the order costs nothing and changes the design. Start with: the model is unavailable. Then: the model is available but unsure. Then: retrieval returned nothing useful. Then: the action succeeded but verification failed. Specify what the system does and what the person sees in each case, and the happy path tends to fall out as the top rung of a structure you have already built.

The ladder, specified

A degradation ladder is only real if three things are written down per rung: what triggers it, what the system does, and what the user is told.

Rung	Trigger	What the user sees
1 · Full answer	Normal	A grounded answer with its sources
2 · Narrower answer	Low-confidence signal on part of the response	Only the claims it can support, with sources
3 · Retrieval only	Generation unavailable, or an output check blocks the answer	The matching documents, framed as here is what we found
4 · Human handoff	Bound exhausted, tier requires approval, verification failed	A clear handover with the Module 5 package, and an honest statement of status
5 · Static response	Everything probabilistic is unavailable	Contact details and known-good links — no model, no retrieval

Three properties make this a structure rather than a list, and Module 5 established the first two: the bottom rung contains no model, and every rung is independently reachable rather than only by descending from the top.

The third is the one this topic adds.

Fallback quality is a specification, not a consequence

Here is the omission that matters.

Teams specify that a fallback exists. They rarely specify how good it has to be — and a fallback nobody specified the quality of will be built to the lowest standard that technically satisfies the requirement.

The difference is enormous in practice. Rung three, done carelessly, is a list of document titles with relevance scores. Done deliberately, it is the three most relevant sections with the matching passage highlighted and a sentence explaining why these were returned. Both satisfy “falls back to retrieval only”. One is useful and the other teaches people that the tool breaks.

So write the quality bar per rung: what the user can accomplish at this level, and what they cannot. That sentence is what stops the degraded path becoming a de facto error page.

graph TD
  A["<b>Rung exists</b><br/><i>the usual requirement</i>"] --> B["<b>Rung is reachable</b><br/><i>entry condition stated</i>"]
  B --> C["<b>Rung is useful</b><br/><i>quality bar specified</i>"]
  C --> D["<b>Rung is exercised</b><br/><i>and therefore works</i>"]

  classDef weak fill:#D8D3CB,stroke:#B8B2A8,stroke-width:2px,color:#3F3F3F
  classDef mid fill:#E8A33D,stroke:#C4832A,stroke-width:2px,color:#ffffff
  classDef strong fill:#7D9A4E,stroke:#5F7A38,stroke-width:2px,color:#ffffff
  class A weak
  class B mid
  class C,D strong

Four levels of specification for the same fallback. Most designs stop at the drained box, which is why degraded modes are so often worse than they needed to be.

Exercise it or it does not work

Module 5 made this point about bounds; it applies with more force here.

Every rung below the first runs rarely, which makes it the least-tested code in the system and the code you depend on at the worst moment. A fallback that has never run in production is a hypothesis.

The responses are ordinary and need to be requirements rather than intentions: fault injection on the model call, a scheduled exercise of each rung, and a small share of traffic deliberately routed to the lower rungs so they stay warm and their quality stays visible. The cost is engineering time on something that produces no feature — which is precisely why it needs to be written down as a requirement, or it will be the first thing cut.

One measurement that costs nothing: record which rung served each request. The aggregate is your early warning. A system silently running on rung three for a week looks healthy on every uptime dashboard and is quietly failing the business case.

What degradation costs

A degraded answer is a worse product experience, and sometimes the honest thing is to prefer a handoff over a poor automated response. That judgement belongs to whoever owns the customer experience, not to you.

And there is a subtler cost: a well-built degradation ladder makes outages invisible, which is good for customers and dangerous for engineering. If nobody notices rung three, nobody fixes what pushed you there. Hence the rung-attribution metric above — it is the countermeasure to your own resilience.

The ladder, with quality bars

Rung	Trigger	What the user sees	Quality bar
1	Normal	Grounded answer with cited documentation	≥ 90% correct on the golden set
2	Partial low confidence	Only supportable claims, cited	Never asserts beyond its sources
3	Generation unavailable, or output check blocks	Top three matching documentation sections, matched passage highlighted, one line of framing	A customer can self-serve from this. Not a list of titles
4	Bound exhausted · tier needs approval · verification failed	Handoff with the full Module 5 package	The agent needs nothing further from the customer
5	Everything probabilistic unavailable	Support contact details and known-good links	Resolvable within one business day via the normal channel

Rung three’s quality bar is the one worth arguing about. It is the difference between a degraded mode that still deflects and one that just annoys. Maya Ortiz owns that bar, because she owns the deflection number — and whether a links-only response counts as a deflection is her call, not an engineering definition.

Rung four’s trigger list includes the retry case from the previous topic: an action whose outcome cannot be verified lands here, with the ambiguity stated. Grace Adeyemi’s team needs that explicitly, because an agent who assumes nothing happened may issue a second credit by hand — the same failure, moved one layer out.

Exercising it. Each rung is exercised on a schedule, with fault injection on the provider call, and the rung that served each request is recorded. The aggregate goes in the monthly report alongside cost per resolved ticket, because a month spent largely on rung three is a month where the business case was not actually being tested.

The honest residual: rung five is a dead end for the customer. It is correct, and it should be rare, and if it is not rare the design has a dependency problem no ladder will fix.

Feeds the workbook

The rubric requires the fallback path to be fully specified — what the user sees when the model is unavailable or unsure. Write the quality bar column; it is the part that distinguishes a specification from a diagram, and the part that decides whether the degraded path is worth having.

→ The ladder needs a trigger. Most of them come from one number nobody calibrates. — Confidence as control flow