vellum-ai QA Mock Backend
Every vellum-assistant client in platform mode needs the same API surface to boot: allauth session → feature flags → consent → orgs → assistants list → daemon (conversations, messages, SSE events, connection-status). Mock that surface and the real frontend lands on the real chat UI with whatever conversation state you script.
Choose a delivery mode
| Mode | When | How |
|---|---|---|
In-process route mocks (Playwright context.route) |
Web testing | Intercept in the browser context; no server process. Recipe in qa-web-ui's harness — endpoint shapes identical to below. |
Standalone bun server (references/mock-server.ts) |
iOS simulator, Electron, any real client binary | Runs anywhere bun runs (host Mac for simulators). Serves the built web dist under /assistant/ AND the mocked APIs from one port. |
Standalone server quickstart
# 0. Probe first — a mock may already be running (leftover from a prior run, or a
# parallel session). Starting a second one just throws a confusing EADDRINUSE.
# A 2xx/3xx/4xx (anything but connection-refused) means one is already up; reuse it.
curl -s -o /dev/null -w "%{http_code}\n" http://localhost:3456/assistant/ || echo "not running — start one"
# 1. Build the web client in platform mode (from a worktree on the branch under test)
cd $QA_REPO_DIR/clients/web && VITE_PLATFORM_MODE=true bun run build
# 2. Run the mock (only if step 0 showed nothing listening; background it appropriately for your shell/tooling)
DIST=$QA_REPO_DIR/clients/web/dist PORT=3456 bun references/mock-server.ts
# 3. Point the client at it
# iOS (capacitor.config.ts): server: { url: "http://localhost:3456/assistant", cleartext: true }
# then re-run cap sync. ⚠️ Check your shell for a VELLUM_ENVIRONMENT export — a
# production value silently wins over the config; force VELLUM_ENVIRONMENT=dev for the sync.
# Info.plist needs NSAllowsArbitraryLoads for local http.
# Web/Electron: open http://localhost:3456/assistant
Pick a port that's free on your machine — 3000 is commonly taken by other dev servers; 3456 is a safe default.
Contract rules (violate these and the app breaks in non-obvious ways)
- Message shape: authoritative
contentBlocks: [{type:"text", text}];contentOrderids must be numeric strings ({type:"text", id:"0"})."text:0"-style ids parseNaNin the reconstruction path → messages render as 24px empty shells. /events/must be a real SSE stream (text/event-stream, initial comment + periodic pings). A 404 destabilizes the session store.- Unmocked daemon subpaths must 404 — a wrong-shape 200 crashes the app's error boundary. Never wildcard-200.
- Assistants list: boot queries
hosting=all(nothosting=platform); answer anyhosting=query.statuslowercase"active",access_consented: true,ingress_urlpointing at the mock's/gwroute with connection-status{state:"ready", is_awake:true}. - Consent field names exactly:
tos_accepted_version/privacy_policy_accepted_version/ai_data_sharing_accepted_version/share_analytics*/share_diagnostics*— wrong names cause a review-terms redirect loop. Version stamps must matchsrc/utils/onboarding-cleanup.tson the branch under test; if you get bounced to review-terms, re-read that file. - allauth session:
meta.is_authenticated: truewith the documented data shape. /conversations/{id}: response must be wrapped —{ conversation: { id, title, created, modified, inferenceProfile: null } }. A bare object crashes the composer settings menu.
Beyond boot: the feature's own dependencies
The contract above boots the app shell — it covers auth, config, and navigation, nothing about the feature you're testing. Every feature has its own endpoints, gating flags, and transports not listed here. Trace the feature's source (generated SDK calls, its api module, flag checks) and extend the server's routes/msgs accordingly before writing assertions — that discovery pass is the real work. Note this server speaks HTTP only: if the feature opens WebSockets or other upgraded transports, fake them client-side (see qa-web-ui's dependency-graph step) or add a Bun.serve websocket handler.
Client-side cache gotchas
- WKWebView (iOS) caches index.html hard — uninstall/reinstall the app in the simulator after changing the served bundle.
- Playwright contexts are fresh by default; no cache issue on web.
Debugging the frontend through the mock
The server's SPA fallback is the injection point: rewrite the served index.html to add a <script> that POSTs to the mock's /debug-log route on window.onerror, unhandledrejection, and console.error, plus a delayed DOM census (message count, rects, computed styles). This is how you see inside a WKWebView with no devtools attached. Beware quote-escaping if generating the snippet through another language.
Scripting test states
Edit msgs in the server (or the route-mock payloads): exact roles, text, ids. Example: an edit-menu test needs exactly one assistant and one user message, to assert the menu appears on assistant text and NOT on user text (always include the negative control).