Migrate

/substrate:migrate

Bring a Gemini AI Studio prototype into the substrate kernel. This is stage 2 — turning a standalone Vite app (frontend only, mock data) into a full-stack Vite + Convex + Clerk project aligned to the three doctrines.

Heads up: this is a multi-minute operation. It discovers the project's doctrines, spawns one doctrine-architect subagent per relevant doctrine in parallel, then executes many file writes with verification gates between each sub-step. Narrate progress so the user sees what's happening.

When to run

• prototype/ directory exists at the repo root (Gemini Build ZIP extracted there).
• Scaffold exists: package.json at repo root + docs/doctrine/ + domain/ + convex/ + test/.
• User has finished iterating on the prototype in AI Studio.

When to REFUSE

Workflow

Step 1. Stage sanity check

Run:

test -f package.json && test -d domain && test -d convex && test -d prototype || echo "STAGE_MISMATCH"

If any check fails, stop and redirect the user.

Verify src/App.tsx still contains the string "Substrate project initialized". If not, migration has already run — ask the user before proceeding (step 5 is destructive to src/).

Step 2. Load the prototype

First, detect the prototype root. The Gemini ZIP extracts with a project-named wrapper folder (e.g. curd-connect/), and /substrate:init tells users to drag that whole folder into /prototype/. So the expected layout is nested:

prototype/<project-name>/package.json
prototype/<project-name>/src/
prototype/<project-name>/vite.config.ts

Resolve PROTOTYPE_ROOT via this logic:

1. If prototype/package.json exists → the prototype is flat. PROTOTYPE_ROOT=prototype.
2. Else if prototype/ contains exactly ONE subdirectory (excluding .gitkeep / dotfiles) and that subdirectory has package.json → the prototype is nested. PROTOTYPE_ROOT=prototype/<subdir>.
3. Else if prototype/ is empty (still just .gitkeep) → the user hasn't extracted the Gemini ZIP yet. Stop and tell them.
4. Else → ambiguous (multiple candidate subdirs or no package.json anywhere). Stop and ask the user which path is the prototype root.

Use $PROTOTYPE_ROOT everywhere below instead of hardcoded prototype/.

Then map the prototype's shape before dispatching architects. Use Glob + Read:

• $PROTOTYPE_ROOT/src/**/*.tsx — components + pages
• $PROTOTYPE_ROOT/src/**/*.ts — utilities, types, mock data
• $PROTOTYPE_ROOT/package.json — reveals which libs Gemini chose (router, state)
• $PROTOTYPE_ROOT/src/App.tsx — entry / layout
• $PROTOTYPE_ROOT/src/main.tsx — bootstrapping (usually trivial)
• Any types.ts / types/*.ts — data shapes
• Any file matching *mock* / *seed* / *sample* / data.ts — mock data sources

You don't need to read every component in full — sample them to understand structure. Pass the resolved $PROTOTYPE_ROOT through to the architect dispatches in step 3 so they look at the correct path.

Step 3. Discover doctrines and dispatch architects in parallel

First, discover the project's doctrines via the same fallback as architect-spec:

1. If docs/doctrine/doctrine-manifest.yaml exists, parse it. For migrate (a one-time bulk operation), default to dispatching every entry whose triggers: plausibly relate to a prototype-migration concern — when in doubt, include. Over-dispatching costs context budget but never produces incorrect migrations.
2. Else glob docs/doctrine/**/*-doctrine.md and dispatch every match.

For each discovered doctrine, spawn its declared specialist (default doctrine-architect) via the Agent tool in a single message with N parallel tool calls. Each gets the migration-focused prompt below, parameterized by the doctrine's path:

Analyze the prototype at $PROTOTYPE_ROOT against your assigned doctrine (path: <doctrine-path>). Identify:

• Concepts and patterns in the prototype that belong in the area your doctrine governs.
• Inline logic in the prototype that violates your doctrine's rules and must be relocated or rewritten.
• Derived properties, validations, or determinism violations within your doctrine's scope.
• For backend-shaped doctrines: mock data sources that should become Convex tables (table names plural camelCase; v.* validators; relationships via v.id(...); indexes inferred from prototype filter/sort sites).
• For frontend-shaped doctrines: components violating pure-presentation, default exports that should become named, routing pattern conversion to TanStack Router, data-fetching sites that should become useQuery hooks in src/hooks/.
• For infra-shaped doctrines: any platform/deployment concerns the prototype implies (env-vars, secrets, external services).

Return your standard output format PLUS a migration file list: for each new file you recommend, list the source path(s) in $PROTOTYPE_ROOT the logic came from and the transformation needed (extract, rename, split, rewrite).

Wait for all dispatches to complete before proceeding. Doctrines whose architects return the "No Recommendations" form indicate the prototype doesn't touch that doctrine's scope — drop them from Step 4's plan rather than synthesizing empty sections.

Step 4. Synthesize the migration plan

Compose the architect outputs into a numbered migration plan. Group sections by each architect's layer-hint, ordered: domain → backend → frontend → infra → cross-cutting. The migration-staging sections (Hooks, Providers, Files dropped) are always present regardless of which doctrines were dispatched — they encode migrate's own workflow, not doctrine output.

Show the plan to the user. For the baseline three-doctrine substrate scaffold, it takes this shape (one extra section per additional doctrine activated, inserted in the layer-hint order above):

Migration Plan — <project name>

Domain (N new files):
  - domain/<file>.ts — extracted from prototype/src/<source>
  - ...

Backend (M tables, K functions):
  convex/schema.ts:
    - <table> {fields} + indexes [<index_names>]
  convex/<feature>.ts:
    - query <verbNoun>: <args> → <return>
    - mutation <verbNoun>: <args> → <return>
  ...

Frontend (P files to move, Q rewrites):
  - prototype/src/components/Foo.tsx → src/components/<feature>/Foo.tsx (named export, remove inline validation)
  - prototype/src/pages/Home.tsx → src/routes/index.tsx (TanStack Router conversion)
  - ...

Hooks (R new bridges):
  - src/hooks/use<Feature>.ts — bridges api.<file>.<fn>, validates via @domain/<file>
  - ...

Providers (src/main.tsx rewrite):
  - Add ClerkProvider + ConvexProviderWithClerk (env vars will be blank until /substrate:deploy)

Files dropped (Gemini AI Studio artifacts not migrated):
  - prototype/package.json, tsconfig.json, vite.config.ts, index.html
  - prototype/metadata.json, AGENTS.md
  - prototype/ itself (archived to prototype-archive/ after migration)

Approve this plan? (y / n / modify)

If the user says n or modify, iterate: ask what to change, re-dispatch the relevant doctrine-architect(s) with the change as additional context, regenerate the plan. Do NOT proceed without explicit approval.

Step 5. Execute the plan

Execute in this order. Verify green between each sub-step. If a sub-step breaks the build, fix it before moving on — don't pile up breakage.

5a. Domain layer. Write domain files per the domain-doctrine architect's recommendations. Write sibling unit tests at test/unit/domain/<file>.test.ts. Run:

pnpm app:compile
pnpm app:test

Must stay green.

5b. Convex schema. Write convex/schema.ts per the backend-doctrine architect's recommendations. Run pnpm app:compile — must stay green. Query/mutation files come next; they import from _generated/ which codegen will produce.

5c. Convex functions. Write convex/_lib/auth.ts (the requireAuth helper from backend-doctrine.md §4.2). Write convex/<feature>.ts files per the plan. These reference ./_generated/* which doesn't exist yet — typecheck WILL fail until npx convex dev runs. That's expected.

Tell the user:

Convex files written. Open a new terminal and run npx convex dev to generate types. Wait for "Convex functions ready!" then come back here and type "continue".

When the user says continue:

1. Patch convex/tsconfig.json so path aliases resolve from the project root:

   bash "$SUBSTRATE_ROOT/scripts/patch-convex-tsconfig.sh"
   

   Convex's generated convex/tsconfig.json ships the path aliases (@/*, @convex/*, @domain/*, @test/*) but omits "baseUrl": "..". Without that, aliases resolve relative to convex/ — @domain/* points at convex/domain/* instead of <root>/domain/*, and every Convex file that imports via an alias fails to typecheck. The patch script is idempotent.

2. Re-run pnpm app:compile. Must now pass.

5d. Frontend migration. Execute these sub-tasks in order to keep intermediate states valid.

1. Move components. For each prototype file in the migration map:

   • Create the new file at its target path with rewrites applied:
     • Convert default exports to named exports (unless it's a route component using createFileRoute).
     • Move hooks out to src/hooks/use<Feature>.ts.
     • Remove inline validation (it's now in domain/).
     • Update imports to use path aliases (@/, @convex/, @domain/).
   • Delete the source file from prototype/src/.

2. Wire TanStack Router in vite.config.ts. The scaffold ships @tanstack/router-plugin as a dep but does NOT activate it. Edit vite.config.ts to add the plugin — it MUST come before react():

   import { TanStackRouterVite } from "@tanstack/router-plugin/vite";
   // ...
   plugins: [
     TanStackRouterVite({
       routesDirectory: "./src/routes",
       generatedRouteTree: "./src/routeTree.gen.ts",
     }),
     react(),
     tailwindcss(),
   ],
   

   The plugin's first run generates src/routeTree.gen.ts (already in the scaffold's .gitignore).

3. Update src/main.tsx to wire the provider tree, gated on env vars being present. If either VITE_CLERK_PUBLISHABLE_KEY or VITE_CONVEX_URL is missing, render the <SetupRequired /> component INSTEAD of calling ClerkProvider (which throws on empty key). This is load-bearing because the user runs pnpm app:dev post-migration — they WILL see this screen until /substrate:deploy wires Clerk.

   SetupRequired ships in the scaffold at src/components/SetupRequired.tsx. Do NOT rewrite it; just import and render.

   import React from "react";
   import ReactDOM from "react-dom/client";
   import { ClerkProvider, useAuth } from "@clerk/clerk-react";
   import { ConvexProviderWithClerk } from "convex/react-clerk";
   import { ConvexReactClient } from "convex/react";
   import { RouterProvider } from "@tanstack/react-router";
   import { router } from "./router";
   import { SetupRequired } from "./components/SetupRequired";
   import "./index.css";
   
   const clerkKey = import.meta.env.VITE_CLERK_PUBLISHABLE_KEY;
   const convexUrl = import.meta.env.VITE_CONVEX_URL;
   const root = ReactDOM.createRoot(document.getElementById("root")!);
   
   if (!clerkKey || !convexUrl) {
     root.render(<SetupRequired />);
   } else {
     const convex = new ConvexReactClient(convexUrl);
     root.render(
       <React.StrictMode>
         <ClerkProvider publishableKey={clerkKey}>
           <ConvexProviderWithClerk client={convex} useAuth={useAuth}>
             <RouterProvider router={router} />
           </ConvexProviderWithClerk>
         </ClerkProvider>
       </React.StrictMode>
     );
   }
   

4. Wire Clerk <SignIn/> and <SignUp/> routes with routing="virtual". When you create src/routes/sign-in.tsx and src/routes/sign-up.tsx, use routing="virtual" (NOT the default routing="path"). Clerk's path routing navigates mid-flow to /sign-up/verify-email-address, which 404s without splat routes; virtual routing keeps the multi-step flow in memory. See docs/doctrine/frontend-doctrine.md §4.2a.

5. Delete src/App.tsx. Route composition now lives under src/routes/ (__root.tsx + per-page files); the top-level App.tsx placeholder is obsolete.

Run:

pnpm app:compile
pnpm app:test

5e. Hooks. Write src/hooks/use<Feature>.ts bridges per the plan. Each hook wraps useQuery(api.<file>.<fn>) + useMutation(api.<file>.<fn>) and returns a minimal named shape ({ posts, isLoading, createPost }). Components already migrated in 5d now import these hooks instead of having mock data.

Run pnpm app:compile && pnpm app:test.

5f. Archive the prototype.

mv prototype prototype-archive

Archive rather than delete, so the user has a reference. Add prototype-archive/ to .gitignore unless the user prefers to commit it for history.

Step 6. Full verification

Run the full green gate:

pnpm app:compile
pnpm app:test
pnpm app:lint

If any step fails, fix the specific issue. Do NOT silently skip failures.

Step 7. Commit

Stage and commit with a structured message:

git add -A
git commit -m "feat: migrate Gemini prototype into substrate kernel

- Moved prototype/src/* → src/ with doctrine alignment
- Extracted <N> domain concepts to domain/
- Drafted Convex schema: <tables>
- Added <M> hooks to bridge Convex to UI
- Wired ClerkProvider + ConvexProviderWithClerk in main.tsx
- Archived original prototype to prototype-archive/
"

Do NOT push. The user decides when to push.

Step 8. Install deps

After committing, pick up any deps added during migration. Run pnpm install (foreground, finite) so the user's dev server starts cleanly in step 9:

pnpm install

This ensures migration-added deps (svix for Clerk webhooks, Gemini-introduced libs like date-fns or lucide-react) are on disk.

Do NOT auto-launch pnpm app:dev — background dev servers linger after the Claude session ends, obscure Vite logs, collide on port 5173 across repeated runs, and surprise the user. Let them start the server themselves.

Step 9. Clerk dev setup

The migration wired src/main.tsx with an env-guard that renders <SetupRequired /> when VITE_CLERK_PUBLISHABLE_KEY / VITE_CONVEX_URL are missing. Fill them in now so the local app actually works.

First, check whether Clerk is already configured (user may have run setup earlier):

grep -q "VITE_CLERK_PUBLISHABLE_KEY=.\+" .env.local 2>/dev/null && echo "CLERK_ALREADY_SET" || echo "CLERK_NOT_SET"

If CLERK_ALREADY_SET, skip to step 10. If CLERK_NOT_SET, invoke the interactive script:

bash "$SUBSTRATE_ROOT/scripts/setup-clerk.sh"

The script walks the user through:

• Creating a Clerk development application at https://dashboard.clerk.com/
• Enabling Email (code / magic link) + optional Google sign-in (Google uses Clerk's shared OAuth in dev — zero GCP setup)
• Creating a "Convex" JWT template in Clerk
• Creating a Clerk → Convex webhook endpoint (URL: https://<your-convex-deployment>.convex.site/clerk-webhook, events: user.created, user.updated, user.deleted)
• Collecting Publishable Key, Secret Key, JWT Issuer Domain, Webhook Signing Secret
• Validating the webhook secret format (^whsec_) before writing anything — common paste error is the URL
• Writing all four values to .env.local
• Running npx convex env set for CLERK_JWT_ISSUER_DOMAIN and CLERK_WEBHOOK_SECRET

If the script exits non-zero, halt and surface the error. npx convex dev must still be running from step 5c for the Convex env-set commands to succeed — if they skip with "run convex dev first", restart it in another terminal and re-run setup-clerk.sh.

Step 10. Local smoke test

Tell the user:

Sign-in check — start the app in a NEW terminal (keep pnpm convex:dev
running from earlier):

  pnpm app:dev

Open http://localhost:5173 and sign in via Clerk (Email or Google).
You should land on an authenticated page with no console errors.

Did sign-in work end-to-end? (y / n) [type 'default' to assume yes]

Wait for explicit confirmation. Common failures:

• auth.config.ts references wrong CLERK_JWT_ISSUER_DOMAIN → re-run setup-clerk.sh
• .env.local was just written but Vite still shows SetupRequired → Vite normally auto-reloads on env changes; if stuck, have user ctrl+c and pnpm app:dev again
• Sign-in redirects to /sign-up/verify-email-address and 404s → ensure <SignIn/> / <SignUp/> use routing="virtual" (see docs/doctrine/frontend-doctrine.md §4.2a)
• No user row in Convex after sign-in → webhook not firing; check the endpoint URL + signing secret in Clerk dashboard match .env.local

If the user types default, proceed as if sign-in worked — they can re-run migrate or fix Clerk directly later.

Step 11. Handoff

Print this summary:

✔ Migration complete. Dev environment fully working.

  Domain:     <N> files, <X> tests passing
  Backend:    <M> functions across <K> tables
  Frontend:   <P> components migrated, routes wired via TanStack Router
  Hooks:      <R> bridges to Convex
  Clerk dev:  connected, sign-in verified
  Convex dev: env vars set
  Prototype:  archived to prototype-archive/

🌐 App live at: http://localhost:5173

Next:
  - /substrate:deploy to take this to production (custom domain +
    prod Clerk + prod Convex + prod env split)
  - OR /substrate:quick-spec to add features before deploying
  - OR /substrate:architect-spec docs/tasks/ongoing/<feature>/<feature>-brief.md for a multi-phase feature

Keep `pnpm convex:dev` + `pnpm app:dev` running in two terminals while you iterate.

Constraints

• MUST stage-check before doing anything destructive (step 1 gate).
• MUST get explicit user approval at step 4 before writing any files (step 4 → 5 gate).
• MUST spawn all dispatched doctrine-architects in parallel — a single Agent-tool message with N tool calls (where N is the number of relevant doctrines discovered).
• MUST preserve doctrine alignment throughout the rewrite: named exports, no hooks in pure components, validation in domain/, v.* validators with indexes, requireAuth on non-public functions.
• MUST run verification (pnpm app:compile && pnpm app:test) after each sub-step 5a→5b→5c→5d→5e. If a sub-step breaks green, fix before moving on — never pile up breakage.
• MUST pause and wait for the user to run npx convex dev at step 5c before typechecking the Convex functions.
• MUST NOT push to GitHub or deploy to Vercel — that belongs to /substrate:deploy.
• MUST NOT invent data shapes not present in the prototype. If a field is ambiguous (e.g. optional vs required), ask the user.
• MUST archive prototype/ rather than deleting outright (step 5f) — the user may want to reference it.
• MUST commit at step 7 so the migration is a single revertable unit.
• MUST auto-run pnpm install at step 8 so the NEXT STEPS commands work cleanly on first try.
• MUST NOT auto-launch pnpm app:dev. Background dev servers cause process-management issues (lingering after Claude exits, port collisions, hidden logs). Print the commands and let the user retain control.
• MUST invoke setup-clerk.sh at step 9 (unless .env.local already has VITE_CLERK_PUBLISHABLE_KEY set — in which case Clerk was configured earlier and we skip). Dev Clerk setup belongs to stage 2, not stage 3.
• MUST pause at step 10 for explicit user confirmation that sign-in works locally. Accept default as assume-yes so the flow isn't blocked by a user who wants to debug sign-in issues out-of-band.
• MUST write src/main.tsx with an env-guard that renders a "Setup required" screen when Clerk/Convex env vars are missing, rather than calling ClerkProvider with an undefined key (which crashes the app).
• SHOULD narrate progress ("architects dispatched", "domain layer written", "verifying", "installing deps", "setting up Clerk dev") — the user is watching a long operation and needs to see liveness.
• Stage-2 end state is a fully working local app with Clerk sign-in functional. "Green compile + tests" alone is no longer sufficient.