Module 4: Start Here

Overview

Modules 1–3 were about the architect’s job: the role, the requirements, the strategy. Modules 4–7 are about the material you are now building with.

This module answers:

What are the layers of an AI system, and what criteria drive the choice at each one?

The AI stack is where the most money is wasted on decisions made by vibes. Model choice, retrieval strategy and serving topology each carry order-of-magnitude cost and latency consequences, and they interact — a model chosen for its context window changes what your chunking strategy should be, which changes your embedding refresh costs, which changes the unit economics you promised a CFO.

A deliberate omission: no model names, no prices

This module names no specific models and quotes no prices, and that is a design decision rather than caution.

Both change on a timescale of weeks. A guide that taught you today’s best model would be teaching you a fact with a short half-life, and worse, it would teach the wrong skill. What survives is the set of criteria that discriminate between options, and the thresholds at which the answer flips — which is Module 3’s criteria-catalog move applied to the fastest-moving layer in the stack.

So where a topic would otherwise cite a number, it tells you which number to go and look up, and what to do with it once you have. Pricing, context windows and capability tiers come from the provider’s own current documentation, and verifying them is part of the exercise.

How the module is built

Two sections. The first chooses the model and the approach; the second builds the infrastructure and the data pipeline that decide whether any of it works.

graph TD
  subgraph S1["Section 1 — The stack and the model"]
    A["1 · Layers of<br/>the AI stack"]
    B["2 · Choosing a model:<br/>criteria over benchmarks"]
    C["3 · The frontier default,<br/>and cascading"]
    D["4 · Vendor lock-in and<br/>the abstraction tax"]
    E["5 · Prompting, RAG<br/>or fine-tuning"]
    F["6 · What fine-tuning<br/>actually costs"]
    A --> B --> C --> D --> E --> F
  end
  subgraph S2["Section 2 — Infrastructure and data"]
    G["7 · Hosted or<br/>self-hosted"]
    H["8 · Inference<br/>economics"]
    I["9 · Throughput and<br/>concurrency"]
    J["10 · Chunking as an<br/>architectural decision"]
    K["11 · Embedding pipelines<br/>and refresh"]
    L["12 · The documentation<br/>quality problem"]
    G --> H --> I --> J --> K --> L
  end
  F ==> G
  L ==> M(["Workbook — Exercise 4"])

  classDef s1 fill:#5B8FB9,stroke:#4A7A9E,stroke-width:2px,color:#ffffff
  classDef s2 fill:#E8A33D,stroke:#C4832A,stroke-width:2px,color:#ffffff
  classDef cap fill:#D9695B,stroke:#B4483A,stroke-width:2px,color:#ffffff
  class A,B,C,D,E,F s1
  class G,H,I,J,K,L s2
  class M cap

The module’s path. Section 1 decides what does the thinking; Section 2 decides whether it can run where you need it and whether its inputs are any good.

Section 1 — The stack and the model

What the layers are → which model → why not the biggest one → what you are locked into → and whether you need retrieval, prompting or training at all.

1. Layers of the AI stackFive layers, and which decisions at each one are expensive to reverse.

2. Choosing a modelThe criteria that actually discriminate, in a form a CFO could audit.

3. The frontier default, and cascadingWhy the strongest model is often the wrong default, and what to do instead.

4. Vendor lock-in and the abstraction taxWhat to abstract, what not to, and what the abstraction costs you.

5. Prompting, RAG or fine-tuningA decision framework based on what is actually broken.

6. What fine-tuning actually costsThe maintenance profile nobody budgets for, and when it is still right.

Section 2 — Infrastructure and data

Where inference runs, what it costs, and the data work that decides whether retrieval is any good.

7. Hosted or self-hostedThe trade-off, and the constraint that usually forces the issue.

8. Inference economicsGPU and CPU, quantization, and what each optimisation costs in quality.

9. Throughput and concurrencyWhy capacity planning here does not resemble planning an ordinary service.

10. Chunking as an architectural decisionThe retrieval choice that is expensive to change and usually made by default.

11. Embedding pipelines and refreshFreshness semantics, re-indexing cost, and handling personal data before it is embedded.

12. The documentation quality problemEvery deflection projection assumes the documentation is good. Check.

How to work this module

About six hours, weighted toward reading:

Read the twelve topics — about 3.5 hours. Section 2 is the half that decides whether the system works, and it is the half that gets skimmed. Resist.
Optional — Workbook exercise 4 — about 2.5 hours. Four artifacts, one of which has to satisfy a constraint you cannot negotiate.

In the workbook

Four artifacts for Lintel:

An AI stack selection with a criteria matrix for each layer
An explicit prompting / RAG / fine-tuning decision with its reasoning
A data and embedding strategy covering the docs corpus and historical tickets, including personal data handling before embedding
A residency-compliant serving topology that satisfies the contracts forbidding data egress

Before you start

Basic retrieval and vector-store mechanics are assumed. What an embedding is, how similarity search works, what a vector index does — all prerequisite. This module is about the architectural consequences: what chunking does to retrieval quality, what refresh does to cost, and which of these decisions you cannot undo cheaply.

If you did Modules 2 and 3, keep your NFR budget and reference architecture open. The cost ceiling and the residency constraint do most of the deciding here, and several topics will tell you to check a number against them.

→ Start with Layers of the AI stack.