The "Just Add RAG" Reflex

Overview

The last of the four antipatterns, and the most common architectural decision made without a diagnosis.

A quality problem appears. Someone proposes retrieval. Nobody has yet said, in one sentence, what is actually wrong.

The reflex

Retrieval is often the right answer — the running case is a retrieval system and correctly so. The antipattern is not the technique; it is the order. The technology is named before the fault is.

Module 4 supplied the diagnostic, and it is worth restating because it is the entire fix:

graph TD
  R["Quality problem<br/>appears"] -->|"the reflex"| X["Add retrieval<br/><i>technology named<br/>before the fault</i>"]
  R -->|"the diagnostic"| Q{"What is<br/>actually wrong?"}
  Q -->|"doesn't know your facts"| A["Retrieval<br/><i>correct</i>"]
  Q -->|"behaves wrong"| B["Prompting"]
  Q -->|"can't do the task"| C["Different model,<br/>or smaller task"]
  Q -->|"corpus is stale"| D["Documentation work"]

  classDef bad fill:#D9695B,stroke:#B4483A,stroke-width:2px,color:#ffffff
  classDef gate fill:#E8A33D,stroke:#C4832A,stroke-width:2px,color:#ffffff
  classDef ok fill:#7D9A4E,stroke:#5F7A38,stroke-width:2px,color:#ffffff
  classDef other fill:#5B8FB9,stroke:#4A7A9E,stroke-width:2px,color:#ffffff
  class X bad
  class R,Q gate
  class A ok
  class B,C,D other

Two routes from the same symptom. Three of the diagnostic’s four branches are not retrieval problems, and no amount of retrieval infrastructure addresses them.

Detection signals

Four, and the first is almost always sufficient:

Nobody can state the fault in one sentence. If the problem statement is “quality isn’t good enough”, the diagnostic has not been run.
The proposal names a technology before a problem. Reverse the order and ask what breaks.
No corpus audit exists. Module 6 established that retrieval amplifies corpus quality in both directions. Proposing retrieval without knowing the corpus can answer the questions is proposing an amplifier with an unknown sign.
The expected improvement is unquantified. “Better answers” is not a target. Module 2’s response-measure discipline applies.

The subtler version

Worth its own warning, because it looks like diligence: “just add more retrieval.”

A wrong answer appears, so retrieval depth is increased — send ten chunks instead of five. This feels like strictly more information and is frequently a net loss. Module 4 explained why: more chunks dilute the match, the genuinely relevant passage competes with marginally relevant ones, and precision drops. Meanwhile input volume rises, so cost rises on every subsequent request.

A change that spends money and may lower quality is a bad trade made with good intentions. The fix is to measure retrieval quality separately from answer quality — which Module 6 required precisely so that you can tell whether the problem is retrieval failing to find the passage or generation failing to use it.

Why the reflex is expensive

Module 4 ranked the layers of the stack by reversal cost, and retrieval infrastructure was the most expensive to change: chunking and embedding decisions are baked into an index that must be rebuilt to alter them.

So the reflex does not merely add a component that may not help. It commits you to the hardest layer to undo, on the basis of a diagnosis nobody performed. That is the architectural argument against it, and it is stronger than the aesthetic one.

The cost of running the diagnostic instead is roughly an afternoon, plus whatever the corpus audit takes. Module 6 priced that audit at a day or two of someone who knows the product.

Correctly chosen, and the assumption still untested

The initiative is a retrieval system, and Module 4 reached that conclusion properly: the diagnostic said the model does not know the product’s facts, the freshness test said those facts change weekly and therefore cannot live in trained weights, and fine-tuning was rejected with reasons and a reconsideration trigger. That is the diagnostic route, not the reflex.

What the reflex version would have looked like, for contrast: a proposal to “add RAG over the docs and tickets” in week one, with no statement of what is broken, no corpus audit, and no decision about what retrieval is expected to fix. It would have produced roughly the same architecture and no ability to defend it — and crucially, no reason to ask the question below.

The assumption that is still untested. The audit says 38% of tickets are answerable from existing documentation. Module 4 and Module 6 both flagged that this is a claim about the corpus, not about the tickets, and that it is testable in a day or two by having someone try to answer fifty of them from the documentation alone.

That test has not been run. Until it is, the retrieval decision is correct in method and unverified in premise — which is a materially different position from either “we checked” or “we didn’t think to ask”, and the honest thing is to say which one you are in.

The live risk of the subtler version. When the first wrong answers appear — and they will — the available responses will include increasing retrieval depth. Module 6’s separate retrieval metrics are what make that decision evidence-based rather than reflexive: if retrieval found the right passage and the answer was still wrong, depth is the wrong lever and raising it will cost money for nothing.

Feeds the workbook

The fourth entry in your antipattern review. The useful question is not whether you chose retrieval, but whether you can state the fault it addresses in one sentence and name the evidence that the corpus can support it.

→ Four antipatterns, all found by inspection. There is a faster way to find them, and it has its own failure mode. — AI as design opponent