Poisoning and the Writable Corpus

Overview

The previous topic assumed an attacker can plant content somewhere the system will later read. This topic asks the obvious follow-up question, and the answer for most retrieval systems is uncomfortable.

It also introduces an attack that is quieter than injection, harder to detect, and more durable.

Your corpus may be a write surface

Module 4’s data strategy indexed two sources: product documentation and historical support tickets.

Documentation is reviewed content with an owner and a change process. Tickets are written by customers. Which means the second half of the corpus is a surface that outsiders can write to, at will, at no cost, with no review — and your system treats what comes back from retrieval as knowledge.

Nobody decides this. It falls out of a reasonable-sounding data decision made for a reasonable-sounding reason (“historical tickets contain useful resolutions”), and the security property arrives silently alongside it.

The general form of the question, worth asking of any retrieval design: who can add content to anything this system retrieves? If any answer includes people outside your organisation, the corpus is part of the attack surface, not part of the trusted base.

Three attacks, in increasing order of quietness

Injection via planted content — the previous topic. Write instructions, wait for retrieval. Loud, in the sense that the planted text usually looks odd if anyone reads it.

Poisoning for persistent wrong answers. Write content that is plausible, well-keyworded, and false. No instructions, no unusual phrasing, nothing that pattern-matches as an attack — just incorrect facts with good retrieval characteristics. Future questions on that topic retrieve it and the system answers confidently and wrongly, with a citation.

Jailbreaks — getting the model to violate its own stated boundaries. Here this is mostly a content-safety and reputational concern rather than an action concern, precisely because Module 5 put authority outside the model. A jailbroken model still cannot exceed a limit enforced in the action layer.

Why poisoning is the nastier one

graph TD
  A["<b>Injection</b><br/>fires once, on retrieval"] --> B["Detectable:<br/>the text looks like<br/>instructions"]
  A --> C["Transient:<br/>one interaction"]
  D["<b>Poisoning</b><br/>sits in the index"] --> E["Undetectable by pattern:<br/>it is just a wrong fact"]
  D --> F["Persistent:<br/>every similar question,<br/>indefinitely"]
  D --> G["Survives model upgrades:<br/>the corpus is what<br/>is wrong"]

  classDef inj fill:#E8A33D,stroke:#C4832A,stroke-width:2px,color:#ffffff
  classDef poi fill:#D9695B,stroke:#B4483A,stroke-width:2px,color:#ffffff
  class A,B,C inj
  class D,E,F,G poi

Two attack profiles. The terracotta branch has no signature to detect, does not expire, and is immune to every improvement you make to the model.

That last branch is the one to sit with. Every other quality problem in this guide gets better when the model gets better. A poisoned corpus does not. You can upgrade the model, tighten the prompt, improve the retrieval ranking, and the system will keep returning the same wrong fact — faithfully, which is exactly what you asked it to do.

Module 4 called retrieval an amplifier of corpus quality with no sign of its own. Poisoning is someone choosing the sign.

Defences, honestly ordered

1. Do not index what you do not need. The cheapest defence is a smaller surface. Module 4 already raised whether the ticket corpus survives redaction with its value intact; this is a second, independent reason to ask. A corpus you do not index cannot be poisoned.

2. Precedence by provenance. Reviewed documentation outranks user-generated content. If a documented answer exists, prefer it, and treat ticket-derived content as supporting rather than authoritative. This is a retrieval-ranking decision with a security purpose, and it costs you some recall on questions only answered in tickets.

3. Resolution-gated inclusion. Index only tickets that a human agent resolved and marked as useful. This is the strongest practical control available here, and it is nearly free, because Module 5 already built the mechanism: the handoff protocol captures human resolutions. An attacker can write anything they like; they cannot cause a support agent to mark it as a good resolution.

4. A delay before indexing. A lag between content being written and becoming retrievable gives detection and review a window. It costs freshness, and for resolved tickets freshness is rarely the binding requirement.

5. Anomaly detection on the corpus. Unusual content patterns, a burst of submissions from one account, text whose retrieval characteristics are suspiciously good. Useful, late, and never complete.

Note what the ordering says: the two strongest controls are what you choose to index and who has to approve it. Both are architectural decisions made at design time, and neither is a filter.

Who can write to this corpus

The answer is: staff at roughly 1,800 customer companies, plus anyone who can open a support ticket. That is the write surface for half the retrieval corpus.

The control that does most of the work: resolution-gated inclusion. Only tickets closed by one of the 45 agents and flagged as a useful resolution enter the index. An attacker can submit a ticket containing a plausible falsehood about scheduling behaviour; they cannot make Grace Adeyemi’s team resolve it and mark it as the canonical answer. The gate is a human decision that already happens, so the cost is a flag in the support tool rather than a new process.

Precedence: documentation outranks ticket-derived content in retrieval. Where both answer a question, the reviewed source wins. The cost is recall on the genuinely useful cases where a ticket resolution covers something the documentation does not — which is, awkwardly, the main argument for indexing tickets in the first place.

Delay: resolved tickets enter the index on the next weekly refresh rather than immediately. Freshness matters for documentation, which is event-driven on publish; it does not matter for a resolution from last Tuesday.

And the finding this produces. Module 4 left open whether the ticket corpus earns its place after redaction. This topic adds a second cost to that column — it is the system’s largest attacker-writable surface — and a third: resolution-gating it means its value depends on support-team behaviour nobody has measured.

The honest recommendation for release one is therefore documentation only, with the ticket corpus as a separately-justified increment carrying its own evidence. That removes the poisoning surface entirely, simplifies the privacy design in the next topic but one, and costs some retrieval recall that nobody has yet demonstrated exists.

Feeds the workbook

Your threat model needs the write-surface question answered explicitly: who can add content to anything the system retrieves. If the answer includes people outside the company, the controls above belong in the document, and the decision about whether to index at all belongs in your Module 4 data strategy.

→ Containment and corpus hygiene are two layers. Here is the full stack, and what each layer costs. — Layered guardrails