QA iOS Simulator (vellum-client-qa)

QA iOS Simulator — vellum-assistant iOS (Capacitor)

Real native verification: actual WKWebView, actual Swift, actual Capacitor plugins, on an iOS simulator. Slower than qa-web-ui — reach for it only when the behavior under test is native.

Requirements

• macOS with Xcode (full, not just CLT), an iOS simulator runtime, bun, xcodegen
• QA_REPO_DIR = a git worktree of vellum-assistant on the branch under test (never test in your main checkout)
• The qa-mock-backend standalone server (this skill's sibling)

One-time setup

iOS runtime: xcodebuild -downloadPlatform iOS (~8GB, once per Xcode major; persists across reboots). Long download — run backgrounded (nohup ... &) and poll rather than blocking a shell timeout.

End-to-end procedure

1. Worktree build: in $QA_REPO_DIR/clients/web: bun install → VITE_PLATFORM_MODE=true bun run build → bun run ios:setup (cap sync + xcodegen).
2. Mock backend: run the qa-mock-backend bun server (DIST=$QA_REPO_DIR/clients/web/dist PORT=<free port> bun mock-server.ts). Script the exact conversation state the test needs.
3. Point the app at it: capacitor.config.ts → server: { url: "http://localhost:<port>/assistant", cleartext: true }, then re-run cap sync. ⚠️ Check your shell for a VELLUM_ENVIRONMENT export — a production value silently overrides the config; force VELLUM_ENVIRONMENT=dev for the sync. Info.plist needs NSAllowsArbitraryLoads for local http.
4. XCUITest target: add to project.yml (including scheme.testTargets), test file under UITests/. Worktree-only — don't commit the target unless asked.
5. Run: xcodebuild test -destination 'id=<sim-udid>' -derivedDataPath /tmp/ios-dd. Get the udid from xcrun simctl list devices available. When you need the app's bundle id (install/uninstall/launch/simctl calls), read it from the built .app's Info.plist (CFBundleIdentifier) rather than assuming it matches capacitor.config.ts — xcodegen can emit a different id than the Capacitor config declares.
6. Video proof: xcrun simctl io <udid> recordVideo <file>.mov during the run, convert via ffmpeg (libx264, -profile:v main -level 4.0 -pix_fmt yuv420p -r 30), deliver.
7. Teardown when done (do it even on failure): kill the mock server by PID (pgrep -fl then kill — not pkill), delete the worktree and /tmp/ios-dd, xcrun simctl shutdown <udid>, and delete the run's video/artifacts once delivered. Confirm nothing is left with pgrep -fl 'mock-server|simctl|ffmpeg'. A failed xcodebuild test must not leave the mock server, simulator, or multi-GB worktree/node_modules behind — that's how disk/fd pressure builds. Check df -h <workspace> before the build; keep the worktree only while its PR is open.

Gotchas (each one cost real debugging time)

1. WKWebView caches index.html hard — uninstall/reinstall the app in the simulator after changing the served bundle.
2. Synthetic XCUITest taps do NOT reliably fire React onClick on WKWebView content. Both coordinate .tap()/.press() and a11y-element taps can resolve onto the button's exact frame and still not trigger the React handler — the web layer never sees a real pointer event. Web text is also not reliably exposed as staticTexts. For anything web-rendered, don't drive it through XCUITest at all: trigger the interaction from inside the web layer by injecting document.querySelector('<selector>').click() into the served index.html (via the mock, see gotcha #4), and time your capture off a beacon log. Reserve XCUITest taps for genuinely native controls (menus, alerts, keyboard) and assert on native artifacts (app.menuItems, springboard.alerts).
3. The notification permission alert appears late and blocks/obscures the screen. It fires after the push-token POST, so it can eat your first tap and sit on top of any screenshot — it dims the whole view with a scrim, ruining color/pixel captures. Do not rely on xcrun simctl privacy grant notifications <bundle-id> (returns "Operation not permitted", re-fires every launch), and do not try to suppress it from the web bundle — the prompt is native (@capacitor/push-notifications requestPermissions()), so gating JS never reaches it. Two ways to clear it, by context:
   • Inside an XCUITest run: addUIInterruptionMonitor + tap Allow on the springboard alert (a direct springboard button tap is most reliable). Accept it, don't just dismiss, so later captures are clean.
   • Driving manually / scripting simctl screenshots without a test target (the common case for one-off visual checks): use the computer-use skill. Recipe: (a) confirm a host_cu desktop client is connected (assistant clients list --capability host_cu — availability can flip between runs, re-check); (b) computer_use_open_app "Simulator" to bring the sim window frontmost — critical: if another window (e.g. the Electron shell) overlaps the sim, the alert's buttons are NOT in the queried accessibility tree; (c) computer_use_observe — the alert's Allow/Don't Allow now appear as AX elements with IDs; (d) computer_use_click by element_id (reliable — CU operates in desktop screen space, so raw coordinates from a simctl screenshot won't map). Then xcrun simctl io <udid> screenshot gives a scrim-free capture. Pitfalls: the first observe may return a stale "Accessibility permission not granted" error — retry, a real click/observe returns the tree; a click can trigger a macOS screen-recording nag that drops and restarts the CU client (its client ID changes — re-fetch from clients list); computer_use_run_applescript can't find the alert (window 1 AX path fails with -1728) — use open_app + element_id instead.
4. Debug visibility inside WKWebView: no devtools attached — inject beacons by rewriting the served index.html (see qa-mock-backend "Debugging the frontend through the mock"). Beware quote-escaping through generator layers.
5. cap sync / xcodegen generate regenerate the project and wipe your rig edits. Every re-sync rewrites Info.plist (dropping NSAllowsArbitraryLoads) and regenerates from project.yml (dropping any UITests target you added by hand). Re-apply the ATS entry and the test target after each sync, or bake them into project.yml so xcodegen reproduces them. Never edit the generated .xcodeproj and then re-sync.
6. All the qa-mock-backend contract rules apply (numeric contentOrder ids, SSE /events/, 404-not-wrong-200, consent field names).
7. Always include a negative control in the test (e.g. menu must NOT appear on the other role's message).

SKILL COMPLETE WHEN

• XCUITest ran green on the simulator with positive + negative assertions
• Video proof recorded, converted, delivered
• Teardown done (mock server killed, simulator shut down, worktree + /tmp/ios-dd removed, artifacts cleaned) — or explicitly deferred with the rig's state noted
• pgrep -fl 'mock-server|simctl|ffmpeg' confirmed empty; no tokens left in scratch