Prompt Injection

Overview

The previous topic established that everything reaching the context is an input, and that the boundary between instruction and data is a convention rather than a mechanism.

Prompt injection is what follows from that. It is the central attack class for systems of this shape, and this topic is going to be unusually blunt about the state of the art.

Two shapes

Direct injection. The user types something intended to override the system’s instructions. Where the system can only answer questions, the damage is bounded — a wrong answer, an inappropriate tone, perhaps a disclosure of the instructions themselves. Where the system can act, it is a path to the action surface.

Indirect injection. Instructions arrive inside content the system retrieves. The attacker never interacts with the system at all; they plant text somewhere that will later be read, and wait.

graph LR
  U["Customer<br/><i>present, identified,<br/>rate-limited</i>"] -->|"direct"| M["Model"]
  A["Attacker<br/><i>absent at the time<br/>of the attack</i>"] -->|"writes content"| C["Corpus"]
  C -->|"retrieved later"| M
  M --> G{"Can retrieved content<br/>authorise an action?"}
  G -->|"no"| S["Wasted response"]
  G -->|"yes"| X["Money, data,<br/>state change"]

  classDef direct fill:#5B8FB9,stroke:#4A7A9E,stroke-width:2px,color:#ffffff
  classDef indirect fill:#D9695B,stroke:#B4483A,stroke-width:2px,color:#ffffff
  classDef gate fill:#E8A33D,stroke:#C4832A,stroke-width:2px,color:#ffffff
  classDef ok fill:#7D9A4E,stroke:#5F7A38,stroke-width:2px,color:#ffffff
  class U direct
  class A,C,X indirect
  class M,G gate
  class S ok

Two routes to the same context. The amber gate is the only thing in the diagram that reliably holds — and it is an architectural property, not a detection problem.

Indirect injection is the architectural one, for three reasons. The attacker is not present, so rate limiting and identity controls do not apply. The content arrives through a path your system trusts by default, because you built it to retrieve things. And the exposure is set by decisions made in Module 4 — what you index, and whether provenance travels with it.

The uncomfortable part

Stated plainly, because the alternative is false confidence:

There is no complete defence against prompt injection. The discipline is mitigation plus blast-radius limitation, not prevention.

The reason is structural. Injection is not a parsing bug with a fix; it follows from the absence of an enforced boundary between instruction and data. Every detection-based mitigation is a blocklist operating against a generator that produces novel phrasings, in a space where the attacker can iterate freely and you cannot enumerate.

Treat this as the current state of the art rather than a permanent law — the field is active. But design as though it is true, because today it is.

What that means for an architect is a reordering of effort. Detection is the weakest layer and gets the most attention. Containment is the strongest and gets the least.

The mitigations, in order of architectural value

1. Reduce the blast radius. Module 5’s action boundary is the only mitigation here that is structurally sound rather than probabilistic. If retrieved content cannot authorise an action — if tenant scope comes from the session, if limits are configuration in the action layer, if consequential actions require confirmation — then a successful injection wastes a response instead of issuing a credit. The attack still succeeds; the consequence is bounded by design.

This gives the governing rule for the whole topic: never let retrieved content authorise an action. Authority comes from the authenticated session, always.

2. Provenance and trust labels. Module 2 required a trust label on every piece of content; Module 4 made it metadata carried at index time. Content below a stated trust level is never treated as instruction. This is genuinely useful and it depends entirely on a decision made at indexing time, which is why it cannot be retrofitted without rebuilding the index.

3. Structural separation. Keep untrusted content in a clearly delimited region of the prompt and instruct the model to treat it as data to consider rather than directions to follow. This measurably helps. It is not a guarantee, and anyone presenting it as one has misunderstood point two of the previous section.

4. Output checks. Catch the consequence rather than the cause. Did the response contain another customer’s data? Did it attempt a tool call the conversation does not justify?

One specific thing to check that is easy to miss: the arguments of tool calls, not just the prose. A classic exfiltration pattern is an injected instruction asking the system to include data in a request to an external endpoint — the data leaves inside a parameter, and an output check that only reads the visible answer sees nothing wrong.

5. Input screening. Pattern detection on incoming content. Worth having as a layer, and the weakest of the five for the reason given above.

What the mitigations cost

Provenance costs an index rebuild if you did not do it up front. Structural separation costs prompt tokens on every request. Output checks cost latency and, where they use a model, money per call on top of the call they protect. Input screening costs false positives, and Module 1 already explained where those lead: a filter strict enough to be safe makes the tool annoying enough to abandon.

And containment — the one that actually works — costs deflection. Every confirmation gate is an interaction that was assisted rather than deflected, which Module 5 priced and which shows up in Section 1’s denominator.

Mitigated, not solved

The exposure: historical tickets are indexed, and the agent can issue credits. An attacker needs only to write a support ticket containing instructions, phrased to be retrieved when a future customer asks a related question.

The containment, which is where the real defence lives. Tenant scope is asserted from the authenticated support session, never from a model-supplied argument. The credit limit is configuration in the action layer. Every credit requires confirmation from one of the 45 agents. Contractual and permission changes are absent from the tool surface entirely. So the realistic worst case from a successful injection is a wasted response or a confirmation request that a human declines — not money leaving.

The layers above it. Ticket chunks carry a trust label of untrusted from Module 4’s indexing decision, and content at that level is never treated as instruction. Untrusted content sits in a delimited region of the prompt. Output checks look for cross-tenant data and for tool-call arguments containing anything resembling customer data.

What Lena Fischer is told, in these words: injection is mitigated, not solved; the residual risk is a wasted interaction rather than a financial or disclosure event; and the reason that is true is the action boundary rather than the filters. A compliance owner who is told “we are protected” stops believing the next thing you say. One who is told “here is the attack, here is why the consequence is bounded, here is what we cannot prevent” can actually assess it — which is the Module 2 point about blockers wanting criteria rather than reassurance.

The honest residual: a confident wrong answer to a customer, caused by planted content, with no money moved and no data disclosed. That is a reputational and quality problem, which makes it the evaluation pillar’s business rather than this one’s.

Feeds the workbook

The rubric requires your threat model to cover indirect injection through ticket content, not just user input. Lead with the containment argument — which actions are reachable and what bounds them — and present detection as the supporting layers.

→ Indirect injection needs somewhere to plant content. For most systems, that place is more open than anyone realises. — Poisoning and the writable corpus