AI as Design Opponent

Overview

The module turns here. Everything until now has been about designing systems that contain AI. This topic and the next are about using AI on your own work.

Four uses, one technique that makes them work, and one limitation that governs all of them.

Ask it to attack, not to review

The single most important thing in this topic.

Asking a model to “review this architecture” produces balanced, agreeable, useless output — a list of strengths, some generic considerations, a closing note that the design looks reasonable. It reads like a review and contains nothing you can act on.

Asking it to attack produces specifics. The framings that work:

“You are a hostile compliance reviewer with authority to reject this. Find the three strongest grounds for rejection.”
“What does this design assume without stating?”
“What is the most expensive thing to change later that this commits to?”
“Here is the business driver and here is the design. Where do they fail to connect?”
“An attacker wants to make this system move money. Walk the shortest path.”

The asymmetry is reliable: a request for balance produces mush, a request for the strongest objection produces something testable. Naming a specific adversarial role — the compliance owner, the sceptical principal engineer, the CFO asking why the unit economics close — produces sharper output than a generic attack, because it constrains what counts as a good objection.

The four uses

graph TD
  A["<b>Red-team your design</b><br/>strongest objections,<br/>unstated assumptions"] --> T["Triage<br/><i>requires domain<br/>knowledge</i>"]
  B["<b>Read an unfamiliar system</b><br/>boundaries, implicit<br/>assumptions, from a repo"] --> T
  C["<b>Recover lost decisions</b><br/>candidate reasons for<br/>undocumented choices"] --> T
  D["<b>Map risks at speed</b><br/>coverage of failure modes"] --> T
  T --> E["The few findings<br/>that are real"]

  classDef use fill:#5B8FB9,stroke:#4A7A9E,stroke-width:2px,color:#ffffff
  classDef gate fill:#E8A33D,stroke:#C4832A,stroke-width:2px,color:#ffffff
  classDef out fill:#7D9A4E,stroke:#5F7A38,stroke-width:2px,color:#ffffff
  class A,B,C,D use
  class T gate
  class E out

Four uses, one mandatory gate. The amber box is where the value is actually created, and it is the step that cannot be delegated.

Red-teaming your own design is the primary use and the workbook exercise. Supply the design, the drivers and the constraints, and ask for attacks. Then triage.

Reading an unfamiliar system — pointed at a repository, a model will characterise the architecture, name the apparent boundaries, and surface implicit assumptions far faster than reading it yourself. It will also be confidently wrong in places. The output is a set of hypotheses to verify, not findings, and treating it otherwise is how you end up briefing a room on an architecture that does not exist.

Recovering lost decisions. Where nothing was written down, a model can propose candidate reasons for why a choice was made — the constraints that would have made it sensible. Only someone who was there can confirm which is true, but a list of plausible rationales is a much better starting point for that conversation than a blank page. Module 9’s ADR-recovery exercise is exactly this.

Mapping risks at speed. Good for coverage — it will enumerate failure modes faster than a workshop. Bad for judgement, because it cannot know which risks matter in your organisation.

The limitation that governs all four

Stated plainly: it produces plausible objections, some of which are wrong. The output needs triage, and triage needs the thing Module 1 called Domain Authority.

So the value is in volume, not accuracy. It raises your floor by catching the things you would have missed through ordinary inattention. It does not raise your ceiling, because the judgement about which findings matter is still entirely yours.

And there is a specific, predictable blind spot worth knowing in advance. It will find:

missing error paths, unhandled states, unstated assumptions
generic security and privacy gaps
places where a stated requirement has no corresponding mechanism
internal inconsistencies between parts of a document

It will miss, almost entirely:

organisational reality. It does not know that a specific principal engineer has an informal veto, that a prior failure poisoned the well for anything resembling it, or that the support team has no capacity for the confirmation volume you just designed.

That second list is the part of architecture Part 4 is about, and it is the part that decides whether designs ship. A red-team pass is a technical floor-raiser, not a substitute for knowing where you work.

A red-team pass, and what it would miss

Running the Modules 4–6 design through a model instructed to attack it, as a hostile reviewer.

What it would plausibly find — and these are real, which is the point:

The entitlement decision is made by a model reasoning about a tool result when it is a decision table. (The hybrid-architectures topic found this; a red-team pass would too.)
The cache holds answers derived from personal data and the erasure path does not reach it. (Module 6’s privacy topic surfaced this as a gap.)
The regression trigger is smaller than the measurement’s resolution. (Module 6’s arithmetic.)
Machine-drafted resolutions could enter the retrieval corpus through resolution-gating. (The hallucination-amplification topic.)
Overflow has no designed behaviour on the action path. (The context topic.)

That is five real findings, and a model would plausibly surface most of them from the documents alone — which is a genuinely useful floor.

What it would miss entirely:

That a machine-learning feature degraded quietly in 2023 and was switched off, so any proposal resembling it meets resistance unrelated to its merits.
That Ivan Petrov holds an informal veto and that the regression gate is partly a political artifact aimed at him.
That Grace Adeyemi’s 45 agents across three time zones may not have the capacity for the confirmation volume the credit gate creates — which is an adoption risk, not a technical one.
That Priya Shah’s two-quarter commitment is already promised externally, making the sequencing non-negotiable in a way no document states.

Four omissions, all organisational, all capable of killing the initiative. None of them is discoverable from the architecture.

So the honest use: run the pass, triage the findings, fix the real ones — and then do the organisational review yourself, because that half is not automatable and it is the half Part 4 exists to teach.

Feeds the workbook

This is the AI-assisted red-team artifact. The rubric asks that it surfaced something you had missed and that you recorded your response — so record the triage too: what it raised, what you rejected and why. The rejections demonstrate judgement as much as the fixes do.

→ Using AI against your own design is one case of a larger shift in what the job is. — From creation to review