The-Utopia-Studio Compound-System Architecture
What it does
Turns "how do we build this?" into a component pipeline, an effort budget, and a set of swap points. The fellow describes the product; the skill returns the architecture as a compound system of specialised parts over a data layer, allocates build effort across Data / Model / Orchestration / Interface, and marks every place a model is called so a new model release is a config change, not a rebuild. It forces the design that most AI products actually need and refuses the one most fellows reach for: one big prompt.
The Icarus reframe
An AI product is not a model with a prompt. It is a compound system — input → router → (retrieve / reason / act in parallel) → validate & guardrails → output — sitting on a data layer, where each part is specialised and separately swappable. The effort split is Data 40 / Model 20 / Orchestration 30 / Interface 10, and the number that stops fellows cold is that the model is only 20%: the work is the data you ground on and the orchestration around the call, not the call. Because the model is one swappable node, every model release becomes a config change gated by an eval — not a reason to rebuild.
When to use / When NOT
Use when: the concept and wedge are set and the question is now how to build v1; a fellow is about to start with "we'll just send it all to a big model"; a build already exists and needs to be decomposed into components with swap points.
| Not this skill | Use instead | Why |
|---|---|---|
| "Write the pass/fail spec — golden cases, failure rate, cost budget" | eval-first-spec |
That defines the scoreable contract for the build; this lays out the components and effort split. The two compose: the swap-point evals here ARE that skill's golden cases. |
| "Draw me the diagram / give me the HTML architecture picture" | design/architecture-diagram |
That renders the dark-themed HTML/SVG. This decides the components and allocation; route the picture there once the pipeline is filled. |
| "Is this defensible / where's the moat" | moat-design-canvas |
That sequences the moat gates. The data layer here feeds the moat, but this skill is a build plan, not a defensibility argument. |
| "Structure the pilot deal / price it" | pilot-six-term-sheet |
That is the commercial term sheet. This is the technical architecture. |
If the input is a bare one-liner with no workflow named, do not invent components. Ask the one question that unblocks it — which workflow's data does this run on? — or return the smallest honest next step.
Method
Copy this checklist:
Compound-System Architecture progress:
- [ ] Step 1: State the one-sentence system job
- [ ] Step 2: Specify the data layer and score its grounding evidence
- [ ] Step 3: Fill the component pipeline (each node: job, I/O, failure mode)
- [ ] Step 4: Allocate effort across Data/Model/Orchestration/Interface
- [ ] Step 5: Name the model-swap points and their gating evals
- [ ] Step 6: Run the kill line before returning
Step 1 — the system job. One sentence: what the whole system turns into what, for whom. Not a feature list. If it needs "and" three times, it is more than one system.
Step 2 — the data layer. The corpus, state, and memory the system reasons over.
Name the source and score it on the evidence ladder: does this data exist and is it
accessible today, or is it a hope? A data layer that is [Hypothesis] ("we'll have
great data once users come") is not designed — it is deferred. The best data layer is
workflow exhaust (see yoda-data-sourcing): data the workflow already produces.
Step 3 — the component pipeline. Fill every row of template.md. The spine is fixed; a product may collapse a node (a rules router instead of a model router) but may not delete one silently.
| # | Component | Its one job | Failure mode if missing or weak |
|---|---|---|---|
| 0 | Data layer | Ground the system in real facts/state | Confident hallucination; nothing to retrieve |
| 1 | Input | Normalise and validate the request; reject malformed early | Garbage in; prompt injection reaches the model |
| 2 | Router | Classify the request and dispatch to the right path (cheap model or rules) | One expensive path for every request; trivial calls cost frontier money |
| 3a | Retrieve | Pull grounding facts from the data layer | Ungrounded answer |
| 3b | Reason | The model call(s) that plan or synthesise | (this is the model — the 20%) |
| 3c | Act | Tool calls, writes, side effects | A read-only toy that cannot do the job |
| 4 | Validate & guardrails | Check output against schema, rules, and policy before it ships | Wrong or unsafe output reaches the user; no autonomy is possible |
| 5 | Output | Format, deliver, and log for eval | Cannot score, cannot improve |
Retrieve / reason / act run in parallel where the router fans out; validate is the gate they all pass through. No node may be "the model handles that" — if the model both reasons and guards its own output, component 4 does not exist and the system cannot be made autonomous.
Step 4 — effort allocation. Map real build work into four buckets. Target and bands:
| Bucket | Target | Band | Rule |
|---|---|---|---|
| Data | 40% | 35–50 | Must be ≥ Model. If Model > Data, that is a red flag on its own. |
| Model | 20% | 10–20 | Above 20% requires a stated, legitimate reason (below), or it is a kill. |
| Orchestration | 30% | 25–40 | Router + retrieve + act + validate. Usually the second-biggest bucket. |
| Interface | 10% | 5–15 | Meet the user in a surface they already use; a new app is rarely the wedge. |
Legitimate reasons for Model > 20%: the product's core value is frontier reasoning (a diagnosis or research engine where the reasoning is the deliverable), or a regulated domain mandates a specific certified/fine-tuned model. Not legitimate, because they are Data or Orchestration problems wearing a Model costume: "we need a great prompt", "our jargon is niche" (that is a glossary in the data layer), "the new model is impressive", "we'll fine-tune to fix accuracy" (fix retrieval and validation first). Name the reason in the plan or push the number back down.
Step 5 — model-swap points. Every model call is a swap point. Fill the swap table:
current model, the swap trigger, the eval that gates the swap, and where the config
lives. A swap point is only real if you can name the eval — otherwise a model change is
a leap of faith, not a config change. Those evals are the eval-first-spec golden cases.
Step 6 — the kill line. Run the self-check at the foot of template.md.
Evidence standard
Money moved 1.0 → behaviour observed 0.7 → artefact shown 0.5 → verbal commitment 0.3 →
opinion 0.1. Here the ladder scores the data layer, because that is where the system
is either grounded or guessing. Do not accept a data layer whose grounding is opinion
(0.1: "there's tons of data in this industry") or a hypothesis (0.1: "we'll collect it
once we launch"). The bar is behaviour (0.7): the data is workflow exhaust the target
already produces, and the fellow can point to a live instance of it. Tag every claim
about data, latency, cost, or accuracy [Fact], [Assumption], or [Hypothesis].
An architecture is done when each component has a named, existing input — not a promised
one.
Gotchas
- The model-first trap. Fellows arrive wanting to pick or fine-tune a model. That is the 20% bucket, chosen before the 40% (data) and 30% (orchestration) buckets exist. Fine-tuning at v1 almost always means Model is over-weighted with no legitimate reason. Route the energy back to the data layer and orchestration first.
- The disappearing guardrail. "The model checks its own output" means component 4 does not exist. A model cannot be its own validator; without an independent guard the product can never move up the autonomy ladder past a human-in-the-loop toy.
- A data layer that is a hope. The commonest way this skill returns "not done" is a
data layer scored
[Hypothesis]. No amount of pipeline elegance saves a system with nothing real to retrieve. Push to workflow exhaust that exists today. - Swap points without evals. Naming the config knob is half the job. If there is no golden-case eval that decides whether the new model is better, the swap is a gamble, and the "model-agnostic" claim is decoration. Point every swap point at an eval.
- Collapsing is fine; deleting silently is not. A rules-based router or a single act step is legitimate for a small tool. Writing the node as "n/a — rules" is honest; omitting it so the diagram looks clean hides a decision.
Examples
examples/sample.md — Mentix's factory-manager AI copilot built as a compound system: the plant-data layer as the 40%, a general LLM behind a swap boundary as the 20%, the router/retrieve/act/validate work as the 30%, chat inside the existing shift tool as the 10%, with three named swap points each gated by golden cases.
Related skills
eval-first-spec— the scoreable contract for this build. Run it alongside; its golden cases are the evals that gate every swap point named here. Architecture without a spec is untestable; a spec without an architecture has nothing to test.design/architecture-diagram— renders the filled pipeline as a dark-themed HTML/SVG diagram. This skill decides the components; that one draws them. Do not restate its design system here.yoda-data-sourcing— how to find the workflow exhaust that becomes the data layer. This skill assumes that layer exists; if it does not, start there.moat-design-canvas— sequences the moat. The compounding data layer designed here is usually the moat's engine, but defensibility is argued there, not here.- Supersedes nothing. It is the build-shape gate at the top of the commit-build stage.