# Databricks Architecture

> Create or edit a Databricks solution-architecture diagram — a Lucidchart-style platform diagram (sources → Lakeflow/Genie pipeline → lakehouse/lakebase → dashboards, Genie, apps → end user, on a governed platform). Use when someone wants to draw, generate, or edit an architecture diagram, visualize a Databricks solution's components and data flow, or produce an architecture image. Produces a self-contained HTML you render to a PNG and iterate on.

- Skill: `databricks-solutions/databricks-architecture` (Agent Skill, multi-file: 8 files)
- Install (CLI): `npx skillmds@latest add databricks-solutions/databricks-architecture`
- Raw SKILL.md: https://api.skillmd.com/api/skills/databricks-solutions/databricks-architecture/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Web & Frontend
- Author: databricks-solutions (https://skillmd.com/u/databricks-solutions)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/databricks-solutions/databricks-architecture

---


# Databricks Architecture Diagram

**Section map** (jump to the one canonical home for each topic):
- *Designing from a request* → **Step 0** read the request's shape · **Step 0.5** where extra context goes (`desc`/`note`/`label`/`ai_reasoning`) · **Step 1** which diagram kind · **Step 2** how much to infer · **Step 2.5** how to draw the data layer.
- *Workflow* → copy a renderer HTML, render to PNG, iterate.
- *The format* → the JSON schema, worked example, **Tabs**, **Positioning** (placement resolution order), **Columns/rowGrid**, **Relational placement**, field tables (**A node** / **An edge**), **Containers**, **physical layout**, **Annotations**, **Custom logos**.
- *Component catalog* → the per-component `wiring:` map (the authoritative edge guide) + icon bank + **Sources**.
- *Authoring rules* → the terse do/don't checklist. *Reference files* → the `.jsonc` examples.

A Databricks solution architecture, drawn to fit **what the request actually describes**. That might be:
- a left→right **flow** (sources → pipeline → compute → dashboards/Genie/apps), the common shape for a single use-case; OR
- a **layered platform** — Databricks added ALONGSIDE an existing stack, or an ask written as explicit LAYERS (ingestion / governance / intelligence / agents / apps) → the layers organize the diagram (usually into lanes); OR
- a **governance / infra** containment picture (UC hierarchy, or cloud/VPC boxes).

**Do NOT force every request into the sources→serve funnel.** The funnel is one shape among several — match the diagram to the request's own structure (see *Designing the architecture* → Step 0). A prompt that lists layers, names external (non-Databricks) systems, or asks for an integration seam wants the **layered** shape, not the funnel.

**The architecture must ultimately make sense.** A user describes what they want in their own words and often leaves out the connective tissue — if they name dashboards and agents but no data layer or compute, ADD what's needed so the story holds together; a diagram missing its ingestion / compute / governance backbone isn't right. Respect what they asked for and keep their emphasis. Equally, don't over-engineer: if a detail wasn't asked for and doesn't help the story, leave it out. Aim for the **smallest architecture that is complete and correct for THIS ask.**

**Overarching principle — fidelity scales with how specific the ask is.** Two goals are in tension: *do exactly what the user said* and *make the architecture make sense*. Which wins depends on how precise and how recent the instruction is:
- **A loose first description** (a use-case, pasted notes, a rough vision — often hand-wavy, sometimes not 100% internally consistent) → your job is **interpretive**: honor the intent and emphasis as much as possible, but fill the gaps and quietly reconcile contradictions so the result holds together. Here *make-it-make-sense* leads. Don't refuse or flag every inconsistency — resolve it into a coherent diagram.
- **A specific instruction** ("connect X to Y", "put the model upstream", "drop the dashboard", "these three are one block") → **respect it literally**, even if you'd have drawn it differently. Precise beats your sense of a tidier architecture.
- **A follow-up correction is the STRONGEST signal there is.** When the user edits or corrects the diagram, do exactly that and do NOT silently revert it toward what seems cleaner to you — if their change looks odd, keep it and record the why in `ai_reasoning`, don't overrule it. Later + more-specific always wins over earlier + vaguer (and over your own defaults).

The rule of thumb: **the vaguer the ask, the more you shape it; the more specific or corrective the ask, the more you obey it verbatim.**

The file is a **flat list of `nodes` + `edges`** — a node is on the canvas iff it's in `nodes` (no visibility/state/diffing). **Author STRUCTURE, not pixels:** place with `col`/`row` lanes, relational fields, `wraps` boxes, and `pin` banners — you almost never write `at`. These compose in a fixed resolution order — full rules in *Positioning* below.

**Edges follow the catalog `wiring:` lines.** Each component's `wiring:` line (in the *Component catalog*) is the authoritative map of what it consumes/depends on: a plain entry is the **normal** edge (draw it when both components are present), one marked *(optional)* is added when the story calls for it. They're the typical connections, not an exhaustive whitelist — a genuine edge the story needs is fine; just don't invent a connection a component wouldn't actually have. **Read the relevant rows before wiring.**

**Avoid floating components.** A **participating** component — one that's IN the data flow — **should** connect to something; by default it doesn't sit on the canvas unwired. `sql-lakehouse` reads from the medallion; `ai-bi-dashboard`/`genie` read from the lakehouse; an app reads from `lakebase`/features and fronts a model; `lakebase` syncs from the pipeline and powers the app. If you place one of these and can't trace an edge into or out of it (following its `wiring:` line), you either forgot the edge or the tile doesn't belong — fix one or the other. The **exception is GLOBAL / cross-cutting things that are ABOUT the whole architecture, not a step in it**: a `type:"note"`/`text` annotation, the `db-platform` banner, a wrapping `box`, and platform-spanning governance (`unity-catalog`/`governance-block`) legitimately have no data-flow edge — they frame the diagram rather than participate in it. But a consumption/app/state tile (`ai-bi-dashboard`, `databricks-apps`, `lakebase`, `genie`, `sql-lakehouse`, a model) is NOT global — it should be wired. This is the failure to watch for in wide multi-lane layouts: those tiles land far from the data they read and get left stranded. **Wiring is a design step, not an afterthought:** after placing tiles, trace the flow end-to-end — walk each source through the pipeline to a consumer/entry point and draw every hop. When a global/governing tile DOES draw an edge, it attaches to what it actually governs — `ai-gateway` to MODEL calls (`model-serving`, apps), `unity-catalog` to data — never chained through whatever tile happens to sit next to it.

## Designing the architecture from a request (architecture-first)

Design a **coherent, functionally-complete architecture that solves the ask** from **the user's request** (their prompt / pasted text) — not from any pre-existing file or default, not a word-for-word transcription, not a fixed template.

### Step 0 — READ THE REQUEST'S OWN STRUCTURE FIRST (do this before anything)

The request usually TELLS you the shape. Extract it before you pick components:

- **Does it list LAYERS / SECTIONS?** ("Layer 1 — ingestion, Layer 2 — governance, Layer 3 — intelligence, …", or numbered/bulleted sections.) → Use the **layered shape** (not the funnel), honoring the request's order and names. Layers often map to left→right lanes, but a layer→column mapping is a starting point, not a rule: a governance/cross-cutting layer usually SPANS the others (a pinned bar/band, not a lane), thin adjacent layers can share a lane, and a layer can run top→bottom. Choose the fewest lanes that keep the flow legible and the connected tiles near each other — many sparse lanes push consumers away from the data they read and leave them stranded. Do NOT collapse a rich multi-layer ask into the 5-tile funnel — that's the #1 failure mode. When the ask *doesn't* dictate its own order, a sane default spine is **sources → data processing → domain + governance → compute → ML / agentic → apps → Genie One** — a starting skeleton only; the ask's own structure and names always win. **Keep each layer coherent** — group same-role tiles into the same lane rather than scattering them (e.g. compute sits as one layer between data and the consumers). This is a preference, not a rule: a layered ask mentioning related tiles in different sentences isn't a reason to split them across lanes unless the ask actually wants them separated.
- **Does it name EXTERNAL (non-Databricks) systems?** (AWS Lambda/Kinesis/S3, an existing app, a third-party bus, "keep X, add Databricks alongside".) → Draw the external side in its OWN boundary `box` and Databricks in ANOTHER; the gap between them is the **integration seam**. Show it **both ways** if the ask mentions callbacks/bi-directional (subscribe IN + call APIs back OUT). Use `file:cloud/<provider>/…` logos for the external side.
- **Does it map future-state to current gaps, name reference customers, or carry other context?** → Those are **`type:"note"` post-its by default** (a `text`/`box` annotation only when you specifically want a plain caption or a titled badge), not core components — place them near what they describe (beside the relevant tile, or below/beside the diagram), just don't let them overlap or crowd the tiles.

**Honor the structure the user handed you** — if they wrote the layers/sections, use those names and order.

### Step 0.5 — where does the EXTRA CONTEXT go? (keep the diagram dense)

A rich request carries far more detail than there are components. Don't drop it, and don't inflate the diagram with it — route each piece to the right lightweight slot, keeping every line **short** so the picture stays scannable:

- **Context ABOUT a component** (what a source/domain/table holds, what a tile is for) → a **`desc`** line under it (title + one short subtitle, like "Lakebase" / "Managed Postgres for app state"). Works on catalog tiles (override only when the default can't say it), and on **`source`** and **`logo`** nodes (set `desc` + it shows; a logo desc is a muted 2nd caption line). **One tight phrase, not a sentence** — the tile stays compact and the layout reserves room for it. If it needs a paragraph, it's not a `desc`.
- **Context from the REQUEST that isn't about any one component** (a rationale, a "keep X / we'll add Y", future-state wins, reference customers, an SLA, a caveat, a team/ownership fact) → a **VISIBLE `type:"note"` post-it, placed beside/below the component it relates to. This is the DEFAULT for request context** — if the user wrote it and it matters, it belongs on the canvas as a note, not compressed into a tile `desc` and NEVER parked in the hidden `ai_reasoning` field. **Lean toward MORE notes:** add one per distinct point rather than cramming several into one or dropping them. (Use a plain `text`/`box` annotation only for a genuine caption/badge; never invent a fake component.)
- **Context about a CONNECTION** (why A feeds B, what flows, an order/step, a non-obvious hop) → a short **edge `label`** drawn ON the edge. Use it whenever the relationship isn't self-evident from position — but keep it to a few words ("Subscribe (streams)", "Query shared context", "Trigger APIs after decision") so edges stay legible and the diagram dense.
- **Your OWN reasoning you want to keep but NOT show** (why you chose a handle, a "don't add X" caution, why a row/col) → the node/edge **`ai_reasoning`** field — it round-trips verbatim and never renders (see rule 10). This is ONLY for your authoring rationale, never for request content the user should see — that goes to a visible post-it above.

The bar: every component and edge that isn't obvious gets a short label or `desc`; everything narrative goes to a note/annotation; nothing becomes a bogus tile, and no line runs long. Dense and self-explanatory beats sparse-but-cryptic or cluttered-with-paragraphs.

### Step 1 — which KIND of diagram is this?

Pick the shape that fits what Step 0 found. Most SINGLE-use-case asks are (1); a multi-layer / "alongside an existing stack" ask is (2).

1. **Flow (solution / demo)** — a left→right DATA-FLOW story: a few sources (default ~4, see *Sources*) → Lakeflow+Genie → lakehouse/Lakebase → dashboard/Genie/app → Genie One → user. The common shape for ONE use-case ("predictive maintenance", "customer 360") or a data+AI feature ask ("Lakeflow Connect + SDP → a model endpoint"). Uses the catalog **tiles** + composites; how much to infer → Step 2; which reference(s) to learn from → *Pick a starting point*. **Layout conventions:** one `box` `wraps` the whole flow (usually not the raw sources) = "the Databricks Platform" (auto-renders behind its children — see *Containers*); `db-platform` + `governance-block` `pin` to its `top-left`/`top-right` (never a raw `at`); Genie One fronts the consumption tiles with auto-arrows.
2. **Layered platform / integration** — the request is organized as **layers**, and/or Databricks sits **alongside an existing (non-Databricks) system**. The layers organize the diagram in the ask's order (usually lanes, but see Step 0 — governance spans, thin layers merge, pick the fewest legible lanes); external systems get their own **boundary box** with an **integration seam** to the Databricks boundary box; governance (Unity Catalog) SPANS the layers as a pinned top bar or full-height band, not one lane tile. Build it from the ask's actual layers (boundary boxes + seam + layers-as-columns). This is the shape for enterprise / "add Databricks to our stack" asks — do NOT reduce it to the funnel. **Logical DOMAINS / knowledge areas** (a "five-domain context", a shared semantic layer, a set of business subject areas any agent can query) → draw each as a `type:"logo"` with **`icon:"file:vendor/genie-ontology"`** (the Genie Ontology mark = logical domain / knowledge), wrapped in a `box` titled for the layer.
3. **Physical / governance** — the Unity Catalog HIERARCHY: workspace → metastore → catalogs → schemas → tables. A **containment** picture (nested boxes), NOT a flow. Trigger: the ask is about UC objects / org structure. Use the container-box presets — see *Databricks physical layout* below.
4. **Infra / networking** — cloud/account topology: VPC / subnets / PrivateLink. Also containment (nested cloud boxes, the *Containers* pattern). Trigger: the ask is about networking / deployment / cloud accounts.

If the ask blends kinds (e.g. "the layered flow, inside our VPC"), compose them.

### Step 2 — (solution/demo only) how much to infer

- **Broad / use-case ask** ("predictive maintenance", "fraud detection", "a governed data platform"): **infer the full end-to-end shape** the use-case implies. Predictive maintenance is a MODEL story → sensors/history → pipeline → ML training → registry → serving → scoring + dashboards. **Don't under-scope** to the couple of nouns typed — add the components that make it actually work.
- **Named-component ask** ("Lakeflow Connect + SDP into UC, served by a model endpoint"): **assemble exactly those into a working whole** — honor what they named + add the connective tissue that wires it (platform `box`, ingest ports, an entry point, the edges). **Don't over-build** past what they asked, **don't under-build** to isolated unconnected tiles.

Either way, components that belong together, wired so the flow reads correctly, on the governed platform.

**Don't copy a reference verbatim — adapt it to THIS ask. References are for inspiration.** Read the reference(s) to learn the PATTERN (which components connect, how the layout/handles/pins work), then build a fresh diagram for THIS ask. **Read as many references as are relevant** — most real asks mix several: take the ML platform's serving lane, add the agent-bricks supervisor, drop the governance bar on top, wrap it in the layered shape's boundary boxes, swap the sources. Pick whichever references inform the ask, learn from each, and compose your own. A reference matches the ask 1:1 only rarely — even then, adapt it (names, sources, which components) rather than reproduce it. See **Pick a starting point** for what each offers.

### Step 2.5 — pick how to draw the DATA / INGEST layer (3 ways)

The bronze→silver→gold data layer can be drawn three ways. Pick by **what the story emphasizes** — don't default to one:

1. **`lakeflow-genie-block` (or `lakeflow-block`) — the big unified block.** Shows the whole ingest story in ONE block: Connect · Zerobus · raw-file landing on the left rail, the SDP medallion (bronze→silver→gold), Delta/Iceberg, and (genie variant) a "Built with Genie Code" footer. **Use when ingestion IS part of the story** — you want to showcase how data lands + gets processed, all products in one tidy block. Optional `bronze_desc`/`silver_desc`/`gold_desc` params add a short caption under each layer.
   ```json
   { "id": "data", "type": "lakeflow-genie-block", "col": "pipeline",
     "params": { "gold_desc": "Business marts + metrics" } }
   ```
2. **`medallion-table` — the simpler block.** Just bronze→silver→gold in one compact tile, no ingest rail. **Use when ingestion is NOT the focus** but you still need the medallion — especially when you want the **Feature Store / Metric Views** forks off gold (`params:{feature_store,metric_views}`, wired via `@out-fs`/`@out-mv`/`@out-gold`). Same `*_desc` layer-caption params.
   ```json
   { "id": "med", "type": "medallion-table", "col": "pipeline",
     "params": { "feature_store": true, "metric_views": true } }
   ```
3. **DIY — compose it yourself.** When you want to go into DETAIL and list the actual tables: a `type:"box"` per layer (title `"Bronze"`/`"Silver"`/`"Gold"`) wrapping `type:"logo"` tiles (icon `bronzeLayer`/`silverLayer`/`goldLayer`, `text` = the table name). Tiles use `col`/`row` inside the box's column; each box needs no `col`/`pad`/size/`z` — it auto-sizes around its tiles (z is automatic — see *Containers*). Flow edges box→box. Optionally wrap all three in a **parent box** titled "Lakeflow Spark Declarative Pipelines" with the SDP logo (`titleIcon:"sdpBrand"`) — a box `titleIcon` accepts ANY icon-library key.
   ```json
   { "id": "b1", "type": "logo", "icon": "bronzeLayer", "text": "orders_raw",   "caption": "right", "col": "bronze", "row": 1 },
   { "id": "b2", "type": "logo", "icon": "bronzeLayer", "text": "events_raw",   "caption": "right", "col": "bronze", "row": 2 },
   { "id": "s1", "type": "logo", "icon": "silverLayer", "text": "orders",       "caption": "right", "col": "silver", "row": 1 },
   { "id": "s2", "type": "logo", "icon": "silverLayer", "text": "events",       "caption": "right", "col": "silver", "row": 2 },
   { "id": "g1", "type": "logo", "icon": "goldLayer",   "text": "customer_360", "caption": "right", "col": "gold",   "row": 1,
     "ai_reasoning": "only add ai_reasoning when there's a real authoring choice to record — don't invent" },
   { "id": "bronze-box", "type": "box", "title": "Bronze", "wraps": ["b1", "b2"] },
   { "id": "silver-box", "type": "box", "title": "Silver", "wraps": ["s1", "s2"] },
   { "id": "gold-box",   "type": "box", "title": "Gold",   "wraps": ["g1"] },
   { "id": "sdp-box", "type": "box", "title": "Lakeflow Spark Declarative Pipelines", "titleIcon": "sdpBrand",
     "wraps": ["bronze-box", "silver-box", "gold-box"], "pad": 28,
     "ai_reasoning": "hidden rationale goes INLINE as an ai_reasoning field — the JSON has no // comments" },
   { "id": "note-residency", "type": "note", "text": "PII stays in the EU region; owned by the Data Platform team.", "below": "sdp-box",  "gap": 32 },
   { "id": "note-sla",       "type": "note", "text": "Data refreshed every 15 min · end-to-end latency ~2 min source→Gold.", "below": "gold-box", "gap": 32 },
   { "id": "note-ownership", "type": "note", "text": "Gold layer owned by the data-insights team.", "rightOf": "note-sla", "gap": 24 }
   ```
   The block is **plain JSON — no `//` comments.** Anything you'd write as a comment about a node is either a visible caption (`desc`/`label`), a visible `type:"note"` post-it, or the invisible `ai_reasoning` field ON that node — never a `//` line. Edges for this block: `bronze-box@r → silver-box@l` and `silver-box@r → gold-box@l` (`flow: true`). Feed any of the three from the left with a few `source` tiles (vary them per demo) — into the ingest ports for option 1 (`@in-lakeflow-connect`/`@in-zerobus`/`@in-direct`), or into `@l` for options 2/3.

   **Add AS MANY `type:"note"` post-its as the request has context for** — one per distinct piece of non-component context (a constraint, an SLA, residency, ownership, a "keep X / add Y" principle, a caveat). The example shows three; a rich prompt may warrant more. Don't cram several facts into one post-it, and don't drop context because there's no tile for it — that's exactly what post-its are for. (Visible post-it vs invisible `ai_reasoning` field: the distinction lives in *Step 0.5* — request narrative the user should see → `type:"note"`; your own hidden rationale → `ai_reasoning`.)

---

<!-- BEGIN: local-render-workflow (stripped when the skill runs inside Solution Builder — the app renders architecture.md live in its own canvas) -->
## Workflow — how to make a diagram

The diagram is **one self-contained HTML file** with its JSON in an inline block. Steps:

1. `cp renderer/architecture-viewer.html my-arch.html` (or `architecture-editor.html` for a browser-editable copy with Load/Download buttons).
2. Replace the JSON inside `<script type="application/json" id="architecture">…</script>` with your array (schema below; plain JSON, no `//`). Learn the schema from **The format** below + the relevant `reference/*.jsonc` — then author your OWN diagram for the ask (don't paste a reference in whole).
3. Open in any browser (no server), or render + read the PNG to iterate (below).

**Render loop (do this every edit):**

```
npx playwright install chromium-headless-shell   # one-time (~90MB)
node renderer/render-arch.mjs my-arch.html        # → my-arch.png
```

**Read `my-arch.png`** — check components present, wired right, laid out clean; fix the JSON; repeat. Uses `chromium-headless-shell` over CDP (just the shell + node 18+, auto-found in Playwright's cache). If it reports "No Chrome/Chromium found", run the install line or set `CHROME_PATH=/path/to/chrome`.
<!-- END: local-render-workflow -->
<!-- BEGIN: in-app-workflow (injected only when the skill runs inside Solution Builder) -->
<!-- END: in-app-workflow -->

---

## Pick a starting point and read trusted example

Which reference(s) each request style maps to (most asks draw from more than one; see *Designing the architecture* above):

| If the user wants… (example prompt) | Start / borrow from | Shows |
|---|---|---|
| A general **data + AI / analytics demo** on a single use-case, or a broad "governed platform / data platform" ask with no named layers | `reference/architecture-complete.jsonc` — a worked **flow** example | The full sources → Lakeflow+Genie → lakehouse/Lakebase → dashboard/Genie/app → Genie One flow. Extend it with what the use-case implies (Step 2). |
| An **enterprise / layered** ask — organized as LAYERS, or "add Databricks ALONGSIDE our existing (AWS/other) stack", or naming an integration seam | *Step 1 shape #2* (no reference file — build from the ask) | Layers-as-columns + an external cloud **boundary box** (Kinesis/Lambda/S3…) beside a Databricks **boundary box**, a bi-directional integration seam, governance spanning the top. Use this the moment the ask has explicit layers or an external system — NOT the funnel. |
| Anything **model-driven** — "predictive maintenance", "churn", "recommendations", "fraud", "ML platform" | `reference/ml-platform.jsonc` | `rowGrid` matrix, medallion Feature Store fork (`@out-fs`), Vector Search / RAG, model training → registry → real-time + batch serving. Predictive-maintenance-shaped. |
| An **assistant / RAG / multi-agent** demo — "route questions across our data + docs + tools" | `reference/agent-bricks.jsonc` | Supervisor over Knowledge Assistant · Genie Agent · Hosted MCPs over a governed medallion → Genie One. |
| A **minimal** "ingest → lakehouse → dashboard + Genie for the business" | *The format* inline example (below) | The smallest ingest → lakehouse → dashboard/Genie → Genie One flow. Use only when the ask is genuinely that small. |
| A **physical / governance** layout — "show workspace, metastore, catalogs, schemas" | `reference/governance-layout.jsonc` (+ *Databricks physical layout* below) | Nested Workspace / Metastore / Catalog / Schema / Table boxes — a governance picture, not a data-flow one. |

You can open multiple to compose them for trusted layout.

## The format

The example below is the **use-case starting point**: when the user types a plain USE-CASE with no named components or layers ("a customer-360 demo", "churn analytics for retail"), start from this funnel and **ADD the components that use-case requires** — a prediction story adds ML training → registry → serving; an assistant story adds a supervisor + Knowledge Assistant; a real-time story adds streaming ingest. A pattern to build ON, not a floor to reduce everything to. (When the ask NAMES layers or an external stack, it is NOT this shape — use the layered reference per Step 0/1.) It also teaches the JSON schema + edges/handles/pins. Emit an array of tabs (`[ … ]`, see *Tabs*). **Plain JSON — no `//` comments.** Any explanation lives INLINE as an `ai_reasoning` field on the node/edge it's about (never rendered, round-trips on save) — the example uses it to teach the schema, exactly as you'd document your own non-obvious choices:

```json
{
  "name": "Customer 360",
  "story": "Ingest our Postgres, ERP, sensor, and PDF data into a governed lakehouse, then give the business a dashboard and a Genie Agent to ask questions in plain language — reached through Genie One, all on Databricks.",
  "columns": ["sources", "pipeline", "compute", "work", "entry"],
  "nodes": [
    { "id": "src-postgres", "type": "source", "col": "sources", "row": 1, "label": "Postgres", "icon": "file:vendor/postgresql",
      "ai_reasoning": "sources stack by row; each source's edge (below) names the Lakeflow ingest PORT it lands on — that target handle drives BOTH the port anchor and the flow animation" },
    { "id": "src-erp", "type": "source", "col": "sources", "row": 2, "label": "Acme ERP", "icon": "text",
      "ai_reasoning": "no vendor logo for this internal ERP → icon:\"text\" draws the label as a brand-colored text badge (niche/internal systems)" },
    { "id": "src-sensors", "type": "source", "col": "sources", "row": 3, "label": "Sensor data", "icon": "sensorSource",
      "ai_reasoning": "realtime stream → its edge targets @in-zerobus (particle-river animation)" },
    { "id": "src-docs", "type": "source", "col": "sources", "row": 4, "label": "PDF documents", "icon": "pdfLogo",
      "ai_reasoning": "files → its edge targets @in-direct (travelling-docs animation)" },
    { "id": "lakeflow-genie-block", "type": "lakeflow-genie-block", "col": "pipeline",
      "ai_reasoning": "the one data-layer block: ingest + bronze→silver→gold, built by Genie Code" },
    { "id": "sql-lakehouse", "type": "sql-lakehouse", "col": "compute",
      "ai_reasoning": "governed serving copy; the consumption lane (dashboard + Genie) reads from here" },
    { "id": "ai-bi-dashboard", "type": "ai-bi-dashboard", "col": "work", "row": 1 },
    { "id": "genie", "type": "genie", "col": "work", "row": 2 },
    { "id": "genie-one", "type": "genie-one", "col": "entry", "rot": 90,
      "ai_reasoning": "business-user entry point / interface onto everything to its left; persona pill built IN (no separate user node); rotated 90° into a slim lane; its edges auto-arrow (no flow/arrow needed)" },
    { "id": "db-platform", "type": "db-platform", "pin": { "at": "top-left", "to": "platform-box" },
      "ai_reasoning": "top-band banner PINNED to the box corner (never absolute at — those drift off-corner when the node set changes); a non-float pin RESERVES a top band so the box grows to enclose it" },
    { "id": "governance-block", "type": "governance-block", "pin": { "at": "top-right", "to": "platform-box" } },
    { "id": "platform-box", "type": "box",
      "ai_reasoning": "one white box wrapping the whole flow = 'all of this is the platform'; no z needed — a wrapping box auto-renders behind its children (see Containers)",
      "wraps": ["src-postgres", "src-erp", "src-sensors", "src-docs", "lakeflow-genie-block", "sql-lakehouse", "ai-bi-dashboard", "genie", "genie-one"] }
  ],
  "edges": [
    { "id": "e1", "from": "src-postgres", "to": "lakeflow-genie-block@in-lakeflow-connect", "flow": true,
      "ai_reasoning": "databases/SaaS land on @in-lakeflow-connect; naming the port on the target handle drives the port anchor AND the flow animation" },
    { "id": "e1b", "from": "src-erp", "to": "lakeflow-genie-block@in-lakeflow-connect", "flow": true },
    { "id": "e1c", "from": "src-sensors", "to": "lakeflow-genie-block@in-zerobus", "flow": true },
    { "id": "e1d", "from": "src-docs", "to": "lakeflow-genie-block@in-direct", "flow": true },
    { "id": "e2", "from": "lakeflow-genie-block", "to": "sql-lakehouse", "flow": true },
    { "id": "e3", "from": "sql-lakehouse", "to": "ai-bi-dashboard", "flow": true },
    { "id": "e4", "from": "sql-lakehouse", "to": "genie", "flow": true },
    { "id": "e6", "from": "genie-one", "to": "ai-bi-dashboard",
      "ai_reasoning": "Genie One fronts the consumption tiles; its edges auto-arrow away from it toward the resource" },
    { "id": "e7", "from": "genie-one", "to": "genie" }
  ]
}
```

### Tabs — the file is an ARRAY of architectures

Top level is a JSON **array**, one element (the shape above) per **tab**; its `name` is the tab label. Multiple tabs = multiple views of one diagram (e.g. "Ingestion" / "Serving"). **Always emit the full array** `[ { … } ]`, even for one tab (a bare object is accepted but re-serialized as an array). Emit every tab each time.

### Positioning: how the strategies compose

You have several placement strategies and they are designed to **layer**, not compete. They resolve in this fixed order — each stage respects everything placed before it, so you can freely combine them:

1. **`at`** (explicit `[x,y]` pixels) — used verbatim, **always wins**. Escape hatch; you rarely write it.
2. **Columns** — `col` (lane) + `row` (order in lane), or a shared **`rowGrid`** matrix. The backbone of most diagrams.
3. **Auto-seed** — a bare node (no `col`/`at`/relational) that is *only* placed by being in some box's `wraps` gets stacked inside that box automatically (see *Containers*).
4. **Relational** — `below`/`above`/`leftOf`/`rightOf`/`alignX`/`alignY` position a node against another node's resolved box (see below). Runs after columns, so the anchor already has its place.
5. **Boxes** — `wraps`/`bounds` size around whatever their children resolved to (columns + relational + seeds), innermost first; a box may itself be placed with `below`/`above`/`alignY`.
6. **Pins** — `pin` docks a banner/persona into a box corner, last of all.

**Precedence per node:** `at` > `pin` > relational > `col`/`row`. Set at most ONE relational field per node.

**Composition cheatsheet** (all verified to work together):
- A `rowGrid` matrix (col+row) **with** a `rightOf` satellite off one tile **inside** a `wraps` platform box **with** a pinned governance banner — all at once. The box grows to include the satellite; the pin reserves its band on top.
- A `wraps` box whose children are a mix of `col`-placed tiles **and** bare auto-seeded ones — the placed tiles keep their lane, the bare ones stack **below** them inside the box.
- Nested boxes (`wraps` of `wraps`), one placed `below` another via a box-level `below` — the whole subtree shifts together.

### Columns, and the optional row-grid

Default placement: **`col` = X** (which lane), **`row` = order WITHIN that lane** (top→bottom); each lane centers its own stack independently. Lanes don't line up row-for-row.

Set top-level **`rowGrid: true`** to turn `row` into a **shared horizontal band across ALL lanes** — same `row` number = same Y line in every column. `col` still sets X; only Y changes. Use it when you want a clean matrix (rows reading straight across):

```
rowGrid: true                columns →   pipeline    ml            serving
                            row 0                     UC Registry   Model Serving
                            row 1   ← (skipped: empty band = vertical space)
                            row 2    Batch Job        Model Training
                            row 3    Medallion        …             Lakebase
```

Rules under `rowGrid`:
- **Rows sit on a FIXED PITCH** — row `N` lands on the same Y line in every column, so tiles across columns register into clean horizontal rows. The whole grid is centered on y=0.
- **`row` numbers are grid coordinates, not just order.** SKIP a number to insert an empty row of vertical space (rows `0,2,4` are more spread out than `0,1,2`) — the lever for opening room when edge labels between two rows collide.
- **No `row`** → that node falls back to stacking within its own `col` (e.g. leave the data sources row-less to just stack them).
- **`at` / `alignY` / `below` / `above` still override** a node's grid position (per node).
- **Line the source up with what it feeds** — give a source the SAME `row` as its target so the feed edge is a clean horizontal line (e.g. `src-pdf` and `knowledge-assistant` both on row 4; `src-postgres` and `medallion-table` both on row 3).
- **A TALL node (medallion with forks, agent-bricks, lakeflow blocks) OVERFLOWS DOWNWARD in its OWN column** — it spans into the next cell's space without pushing other columns or inflating the shared row. So a short tile at the same `row` in another column stays put (no gap from the tall neighbor). Just don't place another node in this column on the row(s) the tall node overflows into, or they'll overlap — give the tall node room below it in its lane.

### Relational placement (place a node against another)

When a node's spot is best described *relative to another node* rather than by a lane, use ONE of these instead of guessing `at`. Each references another node's **id**; it resolves AFTER columns (so the anchor keeps its own place), and the engine auto-de-overlaps the result.

| field | effect |
|---|---|
| `alignX: "<id>"` | copy that node's center **X** (keep your own Y from col/row). |
| `alignY: "<id>"` | copy that node's center **Y** (keep your own X). |
| `leftOf` / `rightOf: "<id>"` | sit just left/right of that node, centered on its Y; `gap` = px between them (default 40). |
| `above` / `below: "<id>"` | sit just above/below that node, centered on its X; `gap` = px (default 40). |

- **One per node.** If several are set only one applies (precedence `alignX > alignY > leftOf > rightOf > above > below`). `at` still wins over all.
- **Chains resolve in order** — `A rightOf B`, `B rightOf C` settles C→B→A.
- **Siblings fan out** — several nodes `rightOf` the SAME anchor spread along the perpendicular axis instead of stacking on one point.
- **A satellite inside a `wraps` box is still enclosed** — the box grows to include it. So `rightOf` a tile that's inside the platform box keeps the satellite in the box.
- **Boxes may use `below`/`above`/`alignY` too** (not leftOf/rightOf) — see *Containers*.

### Top level  *(each element of the tabs array)*
| Field | Required | Description |
|-------|----------|-------------|
| `name` | No | The **tab label** for this architecture. Defaults to "Architecture N". |
| `story` | No | One-line description of this architecture. Metadata (kept in the file); not rendered on the canvas. |
| `options.trademarkLogos` | No | `true` → render real third-party brand logos. Default `false` (neutral badges). |
| `columns` | No | Ordered left→right **lane names**. Nodes reference one via `col`. Add/rename/insert lanes freely for a different shape — no fixed taxonomy. |
| `rowGrid` | No | `true` → `row` aligns across ALL lanes into shared horizontal bands (a matrix). Off (default) → `row` orders within a lane. Full rules in *Positioning* above. |
| `custom_logos` | No | `[{ id, svg }]` — inline SVG logos. Reference one from any node's `icon` as `"custom:<id>"`. See *Custom logos & images*. |
| `nodes` | Yes | The components on the canvas (see below). |
| `edges` | Yes | The lines between them. |

### A node
| Field | Required | Description |
|-------|----------|-------------|
| `id` | Yes | Unique (per tab) free-form handle that `edges`/`wraps`/relational fields reference. Need NOT match `type` (`{id:"gold-medallion", type:"medallion-table"}` is fine). 2nd instance of a component: `type#2` or any unique id. |
| `type` | Yes | The **catalog component id** — this is what identifies the component (label/icon/desc/ports resolve from it): `genie`, `sql-lakehouse`, `lakeflow-genie-block`, `governance-block`, `db-platform`, … (see the catalog below; folds in the old composite "kind") OR a special kind: `source` · `box` · `text` · `logo` · `image`. |
| `col` | placement | The lane (from `columns`) this node sits in. Nodes in a lane stack vertically, centered. **Primary way to place a node.** |
| `row` | No | Within-lane order (default), or a shared cross-lane band with `rowGrid: true` — see *Positioning* above. |
| `wraps` | container | On a `type:"box"`: the node ids this box ENCLOSES. The box auto-sizes around them (+ `pad`, default 24). Nesting works (a box may wrap boxes) — see *Containers*. |
| `bounds` | container | On a `type:"box"`: per-side edge anchors `{ left?, right?, top?, bottom? }`. Each side = `"<nodeId>:<anchor>"` (anchor ∈ `left`/`right`/`center` for x, `top`/`bottom`/`center` for y), or `"col:<name>:<anchor>"` (a lane's edge/midpoint), or `"wrap"`. Lets the box edge cut HALFWAY through a node/column. Unspecified sides fall back to `wraps`. |
| `title` · `titleIcon` | box | On a `type:"box"`: the label drawn on the container's top-left corner (`title`) + an optional icon beside it (`titleIcon` — an icon key like `dbCatalog`, `databricksMetastore`, `file:vendor/databricks`). **Prefer these over `label` for a titled container** (Workspace / Metastore / Catalog / platform boxes). `label` also works but has no icon slot. |
| `pin` | placement | Dock this node into a box corner (overrides `col`). An object `{ at, to?, pad?, float? }`: `at` = one of `top-left`·`top`·`top-right`·`left`·`center`·`right`·`bottom-left`·`bottom`·`bottom-right`; `to` = box id to dock into (default: the largest box); `pad` = inset px (default 16); `float` = `false`/omitted → **reserve a band** (the box GROWS so this never overlaps content — top pin pushes content down, bottom extends the box down), `true` → **overlay** at the corner (may sit over content). Use for banners / personas. |
| `at` | No | `[x, y]` **explicit** position (node center). **Overrides `col`/`pin`.** Use for fully manual placement. (A user drag also persists here.) |
| `size` | No | `[w, h]` if resized from the natural size. |
| `rot` · `scale` · `z` · `pad` | No | Rotation° (0/90/180/270), content scale, stacking order (negative = behind), container padding. |
| `group` | No | A shared string id stamped on several nodes → they form a GROUP: selecting one selects all, and they move together on the canvas. |
| `label` · `icon` | No | Override the catalog default label/icon (only when it differs). **Exception — a `type:"source"` has NO catalog default, so it REQUIRES an explicit `label`** (omit it and the tile falls back to an ugly icon-derived name like "Pdflogo"). `icon` may be a built-in name, a `file:vendor/…`/`file:cloud/…` key, or a `custom:<id>` (see *Custom logos & images*). |
| `ai_reasoning` | No | **AI reasoning — NEVER rendered, never affects layout.** Free text explaining WHY this node is here or what a non-obvious choice means (a relabeled generic tile, why a `row`/`col` was picked, a param's effect). Round-trips verbatim (survives drags/saves). Distinct from `desc` (the visible line) and `type:"note"` (the visible post-it). For your own authoring rationale only — request content the user should see goes to a `type:"note"` post-it. Use it so an example stays self-documenting — see rule 10. |
| `desc` | No | Description line under the label. **On a catalog component: OMIT it**

…(truncated)