Verify Kata Agents

Verify Kata Agents

Use this skill when a change affects the Kata Agents desktop experience. The primary surface is the real macOS Electron app. The repository also contains a headless server, WebUI, and CLI; those have separate Playwright projects and are outside this skill's primary launch path.

Run every command from the repository root. This is a local macOS GUI check: the checked-in Electron harness asserts darwin and does not support a headless Linux run.

Launch

Install and build the desktop prerequisites once:

bun install
bun run ensure:electron
bun run electron:build

The isolated launch-and-proof helper is:

node --experimental-strip-types .agents/skills/verify-kata-agents/scripts/capture-launch-proof.ts

It creates a fresh temporary KATA_CONFIG_DIR, allocates a free Vite port, starts Vite only, launches one Electron process through Playwright, waits for the renderer URL, waits for #root to mount, captures the onboarding/ready screen, and tears everything down. Do not run bun run electron:dev beside an E2E run: that command starts its own Electron instance and can duplicate the backend.

For an existing mapped feature, use the repository's real-Electron runner. It starts and stops the instance for the test:

bun run e2e --grep @smoke --trace on
bun run e2e --grep "persists dark theme mode after reload" --trace on
bun run e2e --grep "Embedded browser panel" --trace on
bun run e2e --grep "@worktree-v2 name root" --trace on
bun run e2e --grep @agent --trace on
bun run e2e --grep @channels --trace on
bun run e2e --grep @memory --trace on
bun run e2e --grep @handoffs --trace on
bun run e2e --grep @katacode --trace on
bun run e2e --grep @approvals --trace on

The dev run is ready when the output includes Vite dev server is ready, Electron renderer window is ready, and renderer #root to mount. A fresh run normally exposes #onboarding-wizard; the provider-free setup path clicks [data-testid="onboarding-setup-later"] and waits for #app-ready. #workspace-picker is the thin-client / missing-wsId gate, not the usual desktop path after Setup later. The @smoke, appearance, structural browser (Embedded browser panel), and @worktree-v2 name root tiers do not require an AI credential. --grep @browser also runs provider-backed annotation-send. --grep @settings also runs Chrome cookie import. @agent, @channels, @memory, @handoffs, @approvals, and browser annotation-send use the configured real provider chain described in e2e/README.md. Authenticated GitHub UAT is a separate --grep "commits, pushes, creates a PR" run, not bare @git.

For a packaged app, set the explicit app path and use the release project:

KATA_E2E_RELEASE_APP="/path/to/Kata Agents.app" bun run e2e:release --grep @smoke --trace on

The release bundle must be locally inspector-compatible; see the desktop-release section of e2e/README.md before using it.

Doctor

Run the read-only doctor against the manifest of the instance that this run owns, while that instance is still alive. The proof helper runs this check internally before cleanup. For a focused Playwright run, select the current run in a second terminal:

RUN_MANIFEST=$(ls -td e2e/test-results/e2e-*/manifest.json | head -1)
node --experimental-strip-types .agents/skills/verify-kata-agents/scripts/doctor.ts "$RUN_MANIFEST"

The doctor refuses an instance it cannot identify as ours. It checks macOS, the root package name/version, the current Git revision, apps/electron/dist/main.cjs, apps/electron/dist/bootstrap-preload.cjs, the manifest's temporary config and artifact directories, the current user's ownership of the manifest Vite listener, and a matching Electron process. The default provider-free diagnosis reports that authentication is not required. Add --agent for an agent tier; it checks that at least one configured provider candidate exists without printing a secret. The live agent flow additionally calls the app's read-only getChatGptAuthStatus("chatgpt-plus") check before configuring the OAuth connection, or validates the selected API-key fallback at setup time.

A doctor failure means the process, build, port, or credential precondition is not trustworthy. Stop driving that instance. Do not attach to a user's ordinary Kata Agents process or a run whose manifest is missing.

Drive

Read features/README.md first, then the feature file for the behavior under test. The stable handles used by the real harness are:

• #root, #onboarding-wizard, #workspace-picker, and #app-ready for lifecycle state.
• [data-testid="onboarding-setup-later"], [data-testid="workspace-create-input"], and [data-testid="workspace-create-button"] for credential-free setup.
• [data-tutorial="new-chat-button"], [data-tutorial="chat-input"], [data-tutorial="model-picker-trigger"], and [data-tutorial="send-button"] for a real agent turn.
• #browser-panel, #browser-annotate-toggle, and #browser-annotation-tray for the integrated browser.
• [data-testid="git-workspace-control"] (click the inner button), [data-testid="git-workspace-new-worktree"], [data-testid="git-workspace-name"], [data-testid="git-workspace-create"], and [data-testid="git-workspace-identity"] for managed workspaces.
• [data-testid="channels-nav"], [data-testid="channel-chat"], [data-testid^="channel-route-"], and [data-testid^="channel-journal-entry-"] for Channel routing.
• [data-testid="bots-nav"], [data-testid="bot-chat"], [data-testid^="bot-memory-"], and [data-testid="bot-memory-context"] for Bot memory.
• [data-testid^="handoff-card-"], [data-testid^="handoff-rail-"], and [data-testid="handoff-rail-result"] for Bot handoffs.
• [data-testid^="task-card-"], [data-testid^="task-rail-"], [data-testid="task-rail-repo"], and [data-testid="task-open"] for Katacode dispatch.
• [data-testid="bot-permission-mode"], [data-testid^="approval-card-"], [data-testid^="approval-deny-"], [data-testid^="approval-allow-once-"], [data-testid^="approval-always-"], and [data-testid^="standing-rule-"] for Bot tool approvals.

Prefer ARIA roles and these markers over coordinates, tab order, generated class names, or DOM position. Drive the user action first and assert the resulting UI and side effect second. The checked-in browser flow uses the Electron webContents adapter only for clicks inside a native BrowserView, which Playwright cannot treat as a normal page; it still exercises the guest page's visible target.

Evidence

The proof helper leaves evidence under its run-specific e2e/test-results/<runId>/ directory:

• manifest.json records the run ID, project, launch target, temporary config directory, Vite port, and artifact root.
• launch-actions.txt records the launch action and observed shell state.
• launch-proof.png is a screenshot with Kata Agents visible.
• launch-proof.aria.yml is the renderer's ARIA snapshot.
• dev-stack-*.log and renderer-console.log preserve process and renderer output.

The Playwright runner writes its JSON report and any --trace on trace under e2e/test-results/ and its HTML report under e2e/playwright-report/. These paths are ignored by Git but survive the run cleanup. Keep the run ID with any report. A valid proof includes the user action, the resulting state, and the relevant side effect:

• launch/onboarding: the renderer mounted and the onboarding wizard is visible; fatal renderer errors are empty.
• appearance: the user-selected mode changes the html class, survives reload, and the namespaced local-storage value reads back as dark/light/system.
• browser: the visible panel retains the same data-browser-instance-id through detach/attach; annotation state is visible in #browser-annotation-tray when exercised.
• agent: the user turn and assistant turn both exist, and the assistant response matches the unique prompt token.
• Git: the UI identity, branch, checkout path, and actual git branch --show-current agree; created remote/managed resources are removed by the test cleanup.
• channels: route rows expose data-route-mode and data-owner-bot-id; journal entries survive restart.
• memory: memory row data-memory-state / data-memory-provenance and context data-memory-ids / cursors / checkpoint revision survive restart.
• handoffs: one [data-testid^="handoff-card-"] with data-handoff-id, rail result text, and the same card after restart.
• katacode: one [data-testid^="task-card-"] with data-task-id, rail repository summary, and the same card after restart.
• approvals: a pending [data-testid^="approval-card-"], deny leaves the unique file absent, allow-once writes it, Explore blocks a later Write, and a standing rule matches only its exact target.

Do not use renderer setters, direct local-storage writes, test-only endpoints, or a final screenshot alone as proof. For external providers, use the existing credential boundary and real provider fallback; do not add a fake production adapter. If a safe path is called a dry run, inspect its files, network, and Git refs before treating it as non-mutating.

Cleanup

The Playwright fixture and capture-launch-proof.ts register the exact Electron and Vite children they start. Their finally cleanup closes Electron, terminates that Vite child, and removes only the temporary KATA_CONFIG_DIR. Never use pkill, killall, or a process-name kill against Electron or Vite. If a failed run strands a process, use the manifest port and the doctor/lsof output to identify only the process owned by this run, terminate that PID, and confirm the port is free.

Cleanup must not remove e2e/test-results/<runId>/, e2e/playwright-report/, or any copied proof artifact. After cleanup, verify that manifest.json, launch-proof.png, and launch-proof.aria.yml still exist. Feature-specific temporary repositories, managed worktrees, branches, sessions, and browser instances must be removed by their fixture; preserve their reports and screenshots.

The harness defaults to one worker. Keep KATA_E2E_WORKERS=1 for this skill: subprocess services around the shared RPC port are not isolated for parallel workers yet, and agent credentials are shared state.

Helpers

These executable helpers are part of this skill:

• node --experimental-strip-types .agents/skills/verify-kata-agents/scripts/capture-launch-proof.ts launches the isolated dev app, runs the doctor while it is live, captures the launch screenshot/ARIA snapshot/action record, cleans up, and verifies the evidence remains.
• node --experimental-strip-types .agents/skills/verify-kata-agents/scripts/doctor.ts "$RUN_MANIFEST" performs the read-only owned-instance/build/port/process check. Add --agent when checking an agent-tier provider precondition.

Use /maintain-verification-skill after adding or changing routes, commands, selectors, or teardown behavior so this map stays aligned with the app.