Exercise 4: The AI Stack

Overview

Four artifacts for Lintel, about 2.5 hours. This is the first exercise whose output could be handed to an engineer as a build specification — and the first where a compliance owner could reject it outright.

Keep the NFR budget from Exercise 2 and the reference architecture from Exercise 3 open. If you came straight to the AI material, the case page carries every fact you need; where a topic references a number from an earlier module, treat it as given.

How to work

In order. The data strategy depends on the approach decision, and the topology depends on everything.

AI stack selection with a criteria matrix per layer — ~45 min
Prompting / RAG / fine-tuning decision — ~25 min
Data and embedding strategy — ~45 min
Residency-compliant serving topology — ~35 min
Self-check — six pass/fail items

Artifact 1 — AI stack selection with a criteria matrix

One matrix per layer, plus a page of reasoning. Five layers: foundation model, orchestration, retrieval infrastructure, serving, observability.

For each layer:

The decision that layer owns, in one line.
Its reversal cost — and mark the one that is genuinely expensive. The rubric requires at least one layer naming a decision that would be expensive to reverse.
The criteria that discriminate, cut to those where serious options differ.
Veto columns separated from scored ones. Residency and data-retention terms eliminate; they do not score.
Evidence per cell, dated. “Measured 2 October against the golden set” beats “4/5”.

For the model layer specifically, state the lever order you intend for cost — free wins, then reduced effort, then narrow-task models, then a cascade — and if you adopt a cascade, name its three costs including split caches.

Do not name a model you have not evaluated. If you have no golden set yet, say what you would measure and refuse to claim a choice is justified until you have.

Artifact 2 — Prompting / RAG / fine-tuning decision

Half a page. Run the diagnostic: what is actually broken?

State the decision, what it optimises for, and what it gives up. Then record the rejected options with real reasons — particularly fine-tuning, with a reconsideration trigger in the bet format: the condition and the evidence that would reopen it.

Include the freshness test explicitly. Knowledge that changes weekly cannot live in trained weights, and saying so in one line pre-empts the proposal arriving again next quarter.

Artifact 3 — Data and embedding strategy

Two pages. The longest artifact, covering two corpora that need different treatment.

Required content:

Per corpus — documentation and historical tickets — the boundary rule, size and overlap approach, and whether you retrieve small and supply large.
The metadata set carried at index time, including tenant, trust label, version or effective date, and source. Note which of these would require a full re-index to add later.
Refresh semantics with a stated staleness bound, and which refresh model satisfies it.
The deletion path — stable identity, vector-to-source mapping, update-as-replace, verification.
Personal data handled before embedding, as a pipeline stage, with what redaction costs you in signal.
An erasure procedure covering primary store, index, caches and traces.
The pipeline re-runnability requirement — idempotent, resumable, able to build a new index beside the live one.
The documentation-quality assumption, stated plainly, with the audit you would run to test it.

Artifact 4 — Residency-compliant serving topology

One diagram plus one page.

The diagram shows where inference happens, where data rests, and where every boundary is crossed. For each crossing, what data moves and under what terms.

The page answers the three gates:

Can inference be pinned to the required region, and does the provider report where it ran?
Do the retention terms satisfy the obligations, as the compliance owner reads them?
Can both be evidenced to an auditor — which makes per-request region recording an architectural requirement, not a contract clause.

Then the capacity arithmetic in two lines: arrival rate × duration = concurrency, and the limit that actually binds. It justifies most of what you chose not to build.

Write it so Lena Fischer could read it and say yes or no. That is the test.

Self-check

Pass or fail.

Model selection uses documented criteria a CFO could audit, not benchmark scores. Evidence per cell, dated, with the measurement method named.
The RAG/fine-tuning decision states what it is optimizing for and what it gives up. With fine-tuning’s rejection carrying a reconsideration trigger.
The residency constraint visibly shapes the topology rather than being noted and ignored. The third gate — evidence — appears as a requirement on the architecture.
Chunking and refresh strategy are specified, not left as “ingest the docs”. Per corpus, with the metadata set and the staleness bound.
PII is addressed at the pipeline stage, before embeddings exist. With an erasure path that reaches caches and traces, not just the database.
At least one layer names a decision that would be expensive to reverse later. With the reason it is expensive.

If something failed

If your matrix has scores but no evidence, you have built a scorecard, not an audit trail. Go back and put the measurement or the “not yet measured” in each cell. Admitting what you have not measured is stronger than a number you cannot source.

If the residency constraint appears only as a note, check for the third gate. Most topologies handle where inference runs and forget that someone has to prove it afterwards — and the proving is what turns a contract into an architectural requirement.

If the data strategy treats both corpora the same, re-read what is actually in the ticket history. Different trust levels, different retention rules and different personal-data exposure are three reasons they are not one corpus.

If you skipped the documentation-quality assumption, that is the most consequential omission available in this exercise. Every number you have used descends from it.

→ The stack is chosen. Next, how much of it to hand to an agent. — Exercise 5, after reading Handbook Module 5.