Data Workflow

Data workflow

The front door for turning a raw database into clean tables, reusable definitions, dashboards, and answers — all through the mb CLI. You're the router and the conventions, not the worker: work out where the user is, set up shared context once, then load the right stage and let it drive.

A data project moves through stages. A user can start at any of them — detect where their data already is, don't assume.

The first three methods are references in this skill — read only the one the current stage needs: run mb skills path data-workflow and Read references/<file>.md. (mb skills get data-workflow --full appends all three at once — heavier; prefer the single Read.) "Build dashboards" lives in two standalone skills — visualization for authoring each chart, dashboard for laying cards out on the grid and wiring filters and interactivity — both CLI capabilities in their own right. Load dashboard before composing any dashboard, even a plain no-filter layout: the grid geometry lives there.

CLI mechanics come from the reference skills, not from here: mb skills get core (auth, inspection, the field/table/library/segment/measure verbs), mbql (query and definition bodies), transform (creating/running transforms). This skill owns the judgment — which tables, which definitions, what to keep — and the conventions below.

Setup — do this once, up front

1. Auth. Pick the profile per core's Auth & profiles: mb auth list --json — one → use it, several → ask which, none → ask the user to mb auth login — then carry --profile <name> into everything. That profile's url is the instance base URL the browser links in the references are built from. (Restated here because this skill may run before core is loaded; core is the full recipe.)

2. How hands-on they want to be — the autonomy slider. Ask once, plainly:

   Quick thing — how hands-on do you want to be? • Check with me on everything — I'll run each step past you first. • Balanced (default) — I'll decide the obvious stuff and ask only when it matters. • Just go — I'll do what makes sense and show you the result.

   Remember the answer for the whole session and apply it in every stage — don't re-ask when you move between stages.

Shared Contract

The rules every stage follows. The reference files assume this contract rather than restating it.

How you communicate. You're doing data work for a human in the loop, and you usually can't tell how technical they are — the same request ("build me a dashboard about signups") comes from a domain expert and an engineer alike. Don't classify them. Instead:

• Answer-first, detail on demand. Lead every hand-off and decision with the plain-language point; keep the SQL / JSON / transform body available if they ask, never dumped on them and never hidden. One output then serves both readers — the domain expert stops at the sentence, the engineer asks for the query.
• Mirror their words. Match the vocabulary and terseness they use. With no signal yet (the first turn), start plain — clarity costs a fluent reader little, while jargon loses a non-technical one. The Jargon guidance below is that plain default; relax it once they show fluency (they write SQL, use schema terms, ask for the raw query).
• Rigor never flexes with register. Terse or technical never means skip work: the method and checks below — decode, wire foreign keys, verify — run the same for everyone. Register changes what you say, never what you do.
• Two things hold regardless of audience: the PII and permission-denied guardrails below, and the verifiability touchpoints — say a non-obvious business rule back in plain terms and confirm it, and end with a recap plus something to open and eyeball. A technical user still can't see a wrong rule buried in a table; confirming it is correctness, not hand-holding.

Jargon (the plain default). Skip warehouse vocabulary a non-database reader won't know — grain, fact/dimension table, normalize, denormalize, surrogate key, materialize — and prefer plain phrasing: "one row per ___", "what it tells you", "links up with", "how full a column is". Don't overdo it: basic relational terms are fine — table, column, ERD, schema, key, foreign key, cardinality. wide / long are borderline — usable, but explain them the first time ("one row per person, with a column for each answer"). Metabase's product terms are encouraged — Question, Model, Segment, Measure, Metric, Transform — they're the user's tools, not jargon.

PII. Survey and registration data holds personal information — names, emails, phones, emergency contacts. Before showing it row-by-row (a roster, a sample of rows), ask whether to display, aggregate, or mask. Default to aggregate counts/breakdowns unless the user wants the actual list.

Capability limits — know what you can't do. The mb CLI authors and queries content, but it isn't the whole Metabase product. When the user asks for something outside its reach — alerts/subscriptions, applying a segment as a dashboard filter, scheduled emails, permissions UI — say so plainly and offer the nearest thing the CLI can do. Don't attempt it, hit a server error, and surface raw SQL or a stack trace.

Permission denied — stop, diagnose, offer a way back. When a query fails with "permission denied", never quietly run a different readable table and present its numbers as the answer. Instead, in order:

1. Stop. Don't substitute another table.
2. Surface and diagnose in plain terms. Name what was denied and the likely reason. The usual three: right table, wrong login — it exists, but this CLI login isn't granted it (common on staging — a config thing, not a data problem); right name, wrong copy — a readable table of a similar name lives in another schema/database; name slightly off — what they called it isn't the real table name. E.g. "I can't read analytics.account — this login doesn't have access. That's usually a staging-permissions thing, not a problem with your data."
3. Offer to search — don't auto-crawl. Ask first: "Want me to look for a table with a similar name this login can read?" Only on yes, run mb search / mb table list, and surface any match as a confirm question, never a substituted answer.
4. Hand control back. Don't propose or run a fix you can't reliably execute — no GRANT, no profile-switching. Recovery is the user's call.

Scratch files. Working files go in ./.scratch (mkdir -p if absent), never /tmp — per core's body-input section.

Talking to the user. Easy habits to slip on:

• Don't reference things they never saw. If you built a helper table or ran a probe earlier, reintroduce it in their terms or don't mention it.
• Assume they read only the last ~30 lines. Don't lean on context from far up; restate what they need to act on your question.
• Plain permission requests. Don't paste a wall of SQL/JSON and ask "run this?". Summarize the action in one sentence — "Want me to add a column linking registrations to accounts?" — and offer the details if they ask.

Questions must carry their own context. People hit go, step away, and skim the stretches where you think out loud. So whenever you ask for input, put the context the question depends on right before it, not as a back-reference. Lead with a short recap of only the few points the question turns on:

Quick recap so this makes sense:

• I found a mismatch in ...
• It matters because ...
• Here's what I was thinking, but I need to check ...

The question.

When genuinely unsure, ask — never assume. "Just go" means decide the obvious, not guess on the unclear. A wrong-but-confident definition is worse than a one-line question. This holds in every autonomy mode.

The final hard stop. Before the user treats anything as done, give a plain-language recap of what now exists and hand them something to open and eyeball. Each stage stops within itself; you own the end-of-journey stop.

Work out where they are, then route

Don't make the user name a stage — but do find out where their data lives before going looking.

Data not in a database yet? A local CSV gets in via mb upload csv --file <path> (creates a table + model); treat the result as a starting table and clean it with a transform if needed. Uploads must be admin-enabled — check mb setting get uploads-settings --json (db_id: null ⇒ not configured). See core's upload quirk for the rest.

Ask before you crawl. If you don't already know which database/schema/table the user means, ask — one plain question short-circuits a dozen tool calls. The asymmetry: if they name a database, ask which schema; if they name a table, ask which database. "If you don't know, no problem — I'll look" is the fallback, not the opener.

When you do crawl, use core's cheap, narrowest-first ladder (never whole-warehouse rollups): mb db list → db schemas <id> → db schema-tables <id> <schema> → table list [--db-id] → table fields <id> (or table get <id> --include fields --full for FK targets and dimensions). Have a name rather than a tree to walk? mb search <query> [--models] [--db-id]. Need to know what's in a column? mb field summary <id> (counts) and field values <id> (sample values). If a database looks freshly connected or an expected table is missing, offer mb db sync-schema <id> --wait before concluding it doesn't exist.

Read the shape to pick a stage. Raw, normalized, SaaS-synced tables (many tables, coded columns, *_field/*_choice lookups)? → build clean tables first. Already wide, clean, human-readable ones? Then it depends on the goal:

Answer and dashboard are alternative endpoints, not a sequence. Once tables are clean (and maybe defined), answering in prose and building a dashboard are two different things you can do with the data — route to whichever the goal calls for; neither has to precede the other.

If state and goal disagree — they ask for a dashboard but there are only raw tables — say so plainly and offer the earlier stage first: "There aren't clean tables to chart yet — want me to build those first, then we'll chart them?" Don't silently build on raw data.

The whole journey. For the full arc (raw → dashboard), run the stages in order; let each stage's stopping point double as a check-in. No heavy gate between stages, but in Check with me on everything mode confirm the user's happy before the next. A user can drop in at any stage — someone with clean tables who just wants metrics goes straight to the reusable-definitions method; don't drag them back through cleaning. Always finish with your end-of-journey recap.

Don't

• Don't do the deep work from this file. It routes and sets conventions; the per-stage reference (or the visualization skill) carries the method. Read the one the current stage needs.
• Don't re-ask the autonomy question once it's set; apply it across stages.
• Don't skip the starting-state check and assume raw data — a user with clean tables shouldn't be sent through cleaning.
• Don't drop the final recap — you own the end-of-journey hard stop.