# QA Web UI (vellum-client-qa)

> Visually test vellum-assistant clients/web changes on a feature branch: boot the app locally with mocked platform APIs, drive it in headless Chromium with mobile/coarse-pointer emulation, dispatch real CDP touch gestures, and produce screenshots plus an annotated mp4 video as proof. Use for swipe gestures, mobile layouts, safe-area/iOS-flavored UI, or any before/after visual verification of a branch.

- Skill: `vellum-ai/qa-web-ui-vellum-client-qa` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add vellum-ai/qa-web-ui-vellum-client-qa`
- Raw SKILL.md: https://api.skillmd.com/api/skills/vellum-ai/qa-web-ui-vellum-client-qa/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: vellum-ai (https://skillmd.com/u/vellum-ai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/vellum-ai/qa-web-ui-vellum-client-qa

---


# QA Web UI — vellum-assistant clients/web

Boot the web app from any branch, mock the platform APIs (see `qa-mock-backend`) so it lands on the real chat UI, drive it with headless Chromium under mobile emulation, and capture screenshots + an annotated mp4 as proof.

## Environment assumptions

- `QA_REPO_DIR` = your vellum-assistant checkout (use a worktree if the main checkout is mid-work)
- `QA_HARNESS_DIR` = a scratch dir for test scripts (e.g. `/tmp/qa-web`), with `bun add playwright-core` done there
- Chromium via `bunx playwright install chromium`. The executable path differs by platform: Linux `.../chromium-*/chrome-linux64/chrome`; macOS it's inside the app bundle — `.../chromium-*/chrome-mac*/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing` (`--dry-run` doesn't reliably print it on macOS; `ls` under `$PLAYWRIGHT_BROWSERS_PATH` and dig)
- The harness must run in the same network namespace as the dev server. If your assistant runs in a sandbox/container and the dev server runs on the host, the sandbox's `localhost` is NOT the host's — run the harness on the host, or use the container gateway IP
- One `chromium.launch` per bun process — structure multi-scenario runs as one script, not parallel launches

## ⚠️ Resource budget — check BEFORE you clone or install

This rig is disk- and file-descriptor-hungry: each dev server + browser context holds many fds/sockets, and each checkout's `node_modules` is multi-GB. Left unchecked it exhausts disk/inodes/fds and turns a small QA task into a multi-hour incident.

- **Check disk first:** `df -h <workspace>` before any clone/install. Rethink or clean up if it would pass ~70%; hard-stop at ~80%.
- **One clone, many worktrees — never N fresh clones.** Add per-branch worktrees off a single checkout (`git worktree add`); worktrees share git objects for free. You still `bun install` per worktree (deps must match the branch — see Step 1), but that cost is unavoidable either way, so a worktree is strictly cheaper than a full clone.
- **Cap concurrency at 2–3** live browser/dev-server sessions; serialize the rest.
- **Never write tokens/secrets to scratch files.** Mint short-lived tokens on demand and keep them in ephemeral `/tmp`, not the persistent workspace.

## What can't be tested here (say so honestly)

Real WKWebView quirks, keyboard `visualViewport`, real safe-area insets (simulate by setting `--safe-area-inset-top` on a wrapper), native edit menus/plugins → `qa-ios-simulator`.

⚠️ **Test the real booted app, not a synthetic component mount.** It's tempting to shortcut a layout check by importing one component and rendering it into an ad-hoc root. Don't — if the app builds its styles at runtime (e.g. Tailwind v4 injects into `adoptedStyleSheets` rather than static `<style>`/`<link>`), a component rendered outside the app's own bootstrap gets **no styles at all** and every layout assertion is meaningless (elements read `position: static`, zero geometry). This whole skill is "boot the real app + drive it" precisely because that's the only context where styles, providers, and stores exist. Verify by checking a known app element is actually styled (e.g. computed `position`/`display` is non-default) before trusting any geometry.

## Step 1 — Branch & dev server

```bash
cd $QA_REPO_DIR && git fetch origin <branch> && git checkout <branch>
cd clients/web
VITE_PLATFORM_MODE=true setsid nohup bun --bun vite --port $QA_PORT > /tmp/vite.log 2>&1 < /dev/null & disown
sleep 10 && curl -s -o /dev/null -w "%{http_code}\n" http://localhost:$QA_PORT/assistant/
```

⚠️ `VITE_PLATFORM_MODE=true` is mandatory — without it the app boots local mode, ignores every mock, and dead-ends at `/assistant/onboarding/hosting` (truthy: `1`/`true`/`yes`, see `src/lib/local-mode.ts`). Pick a free port. If the tree is dirty, stash or ask first.

⚠️ **Deps must match the branch under test.** A worktree or checkout that shares/reuses another branch's `node_modules` will crash on boot when the test branch (or main) has added packages or generated-SDK members the stale install lacks — a monorepo makes this common. Symptom: import/export-not-found errors before the app paints. Fix: run `bun install` at the checkout root so it resolves against that branch's lockfile, and regenerate the SDK (`bun run openapi-ts`) if the branch changed the API surface. When testing an uncommitted diff, add a worktree for the base branch (`git worktree add`) and `bun install` there, then apply the diff — don't graft onto a sibling's deps, but don't spin up a whole fresh clone either (see Resource budget above).

⚠️ Shell gotchas: `pkill` can kill your own tool shell — `pgrep -fl` then `kill <pid>`. Some sandboxes can't `file_write` to /tmp — use `cat > file <<'EOF'`.

## Step 2 — Boot mocks

Full copy-paste script in `references/harness.md`. The endpoint set and payload shapes are the `qa-mock-backend` contract delivered via Playwright `context.route` (context-scoped survives navigations). Success = final URL `/assistant/conversations/<id>` with the hamburger (`[aria-label="Open navigation"]`) present. Failure diagnosis table at the bottom of the harness.

## Step 2b — Map the feature's dependency graph (required — budget real time for it)

The base mocks land you on the app shell; they test **nothing** about your feature. Every feature has its own API surface, gating conditions, and transports that are NOT in the base contract. Before writing assertions, trace the feature's source:

- **Endpoints:** find the generated-SDK calls / fetch paths the feature hits (grep `src/generated/*/sdk.gen.ts` and the feature's api module), mock them per the qa-mock-backend contract rules.
- **Gating:** feature flags, consent state, platform checks (`isNativePlatform`, pointer-coarse) that decide whether the UI renders at all.
- **Transports:** `context.route` intercepts **HTTP only** — WebSocket and EventSource upgrades pass through untouched. Real-time features need an `addInitScript` that replaces `window.WebSocket` with a fake that transitions to OPEN (and stays), scripting incoming messages if the UI waits on them.

This discovery pass is where most of the testing time actually goes — treat it as the work, not overhead.

### Route-mocking gotchas

- **Glob `*` does not match `/`** — `**/v1/items/*` silently fails on paths with deeper segments. Use `**` or function matchers (`(url: URL) => boolean`) for anything with subpaths.
- **Scope patterns to the API path structure, never a keyword.** The Vite dev server serves your API mocks AND the app's source modules from one origin — a broad glob like `**/events/**` also intercepts TypeScript modules with "events" in the filename. Symptom: blank page, no obvious error.
- **Unmatched requests must pass through, never hit a generic catch-all.** No handler = Playwright passes through (correct for source modules/HMR); inside a shared handler use `route.fallback()`/`route.continue()` for paths you don't own. A wrong-shape `200 {}` is worse than a 404 — shape-validating code crashes and the error boundary can't tell it from a real response. The one deliberate exception: unmocked daemon API subpaths should 404 per qa-mock-backend contract rule 3 (blanket pass-through there would hit a real network fetch and hang).

## Step 3 — Populated conversations (transcript content)

Two routes:
- **Network path (preferred, matches production):** mock `GET **/v1/assistants/**/messages` per the `qa-mock-backend` contract — served from origin root `/v1/...`, NOT under `/gw/` (only connection-status rides gw). Numeric contentOrder ids, wrapped `/conversations/{id}` response.
- **In-page seeding (when you need store-level control):** dynamic-import `history.ts` + `chat-session-store.ts` and call `seedSnapshot` — recipe in `references/harness.md`. ⚠️ **The seed only works if it writes to the *same* store instance the app is rendering from.** Two things break that: (1) Vite HMR after a source edit — restart the dev server first; (2) dynamic-importing the store module can resolve a *different* copy of it than the running bundle (module-graph split), so your `setState`/`seedSnapshot` mutates an orphan instance and the UI silently never repaints. If a seed appears to no-op with no error, that's the tell — prefer the network path (mock `messages`) so state flows through the app's own entry, or confirm identity by writing a sentinel and reading it back from a store selector the app subscribes to.
- Simulate iOS native: `window.Capacitor.isNativePlatform = () => true`, then reseed with **fresh message ids** to force remount (`useIsNativePlatform` only re-reads on fresh mounts).

## Step 4 — Mobile emulation & touch

- Context: `{ viewport:{width:390,height:844}, isMobile:true, hasTouch:true, deviceScaleFactor:2 }` — verify `matchMedia('(pointer: coarse)').matches === true` or gesture code never arms.
- Touch: `context.newCDPSession(page)` + `Input.dispatchTouchEvent` (touchStart → stepped touchMoves → touchEnd with empty touchPoints).
- Always read gesture constants on the branch under test (e.g. `EDGE_ZONE_PX` in `use-edge-swipe.ts`) before writing assertions.
- ⚠️ Over iframes CDP re-hit-tests each move with iframe-local coords — set the iframe `pointer-events:none` during the gesture.
- Assert from the DOM (aria-expanded, `location.pathname`), never screenshots alone. Always include a negative control.
- ⚠️ Don't assert on animation state attributes (`[data-state="open"]`) for animated components — they can settle after the transition completes or live on a non-visible ancestor. Assert on geometry (bounding-rect position/size) or `elementFromPoint` at the element's center instead.
- ⚠️ For anything trigger- or position-dependent (popovers, selection-anchored UI, drag targets), drive it with the **real user gesture**, not a synthetic store call with a hardcoded rect. A direct `openThing({ anchorRect })` produces geometrically valid but contextually fake output — the element renders somewhere plausible but detached from what would actually summon it, which reads as a bug in screenshots. If you must inject synthetically to isolate a layout check, say so in the QA output so a stray anchor position isn't mistaken for a real defect.

## Step 5 — Video proof

`recordVideo: { dir, size:{width:390,height:844} }` on the context; annotate with a red touch-dot div + test-label div, slow the gesture for the camera (see harness). ⚠️ `context.close()` before `browser.close()` or the video is corrupt. Convert with a libx264 ffmpeg:

```bash
ffmpeg -y -i <video>.webm -c:v libx264 -profile:v main -level 4.0 -pix_fmt yuv420p -r 30 -fps_mode cfr -movflags +faststart out.mp4
# -profile:v main -level 4.0 -r 30 REQUIRED for QuickTime compatibility
```

Read your screenshots yourself to verify before claiming results.

## Step 6 — Cleanup (mandatory, runs even on failure)

Cleanup is part of the task, not an afterthought — a thrown assertion must NOT leave a browser or dev server running (that's how fds pile up). Structure the harness so teardown is failure-safe:

- **In the harness script, wrap the run in `try/finally`** so `context.close()` → `browser.close()` always fire even when an assertion throws (`context.close()` before `browser.close()` or the video corrupts — see harness). Never leave a context/CDP session open.
- **Kill the dev server by PID** (`pgrep -fl vite` then `kill <pid>` — not `pkill`, which can kill your own shell) and confirm nothing is left: `pgrep -fl 'vite|chrome|playwright|ffmpeg'` should be empty.
- **Delete this run's artifacts once delivered** — screenshots, `.webm`/`.mp4`, and any per-run temp/scratch dirs. Upload to the PR first, then remove locally, same session.
- **If you created a clone or worktree just for this run, remove it** (`git worktree remove`, incl. its `node_modules`) once done. Keep worktrees only while their PR is open.
- `git checkout --` any diagnostic instrumentation and return the repo to the prior branch.

## SKILL COMPLETE WHEN

- [ ] App booted to `/assistant/conversations/<id>` (not onboarding) — final URL logged
- [ ] At least one positive and one negative assertion, results from DOM state
- [ ] Artifact delivered (screenshot/video)
- [ ] Dev server killed by PID; `pgrep -fl 'vite|chrome|playwright|ffmpeg'` confirmed empty
- [ ] Run artifacts + any run-only clone/worktree removed; repo on the expected branch
- [ ] No tokens/secrets left in scratch