User Acceptance

User Acceptance Evidence

Operating brief

When this skill triggers, gather real evidence that the feature branch, ticket, or PR works. Treat the request as: "prove the completed work by exercising it and producing artifacts a human can inspect."

Acceptance evidence should be experiential and reproducible:

• For UI changes, capture video when practical, then screenshots at meaningful checkpoints. Video captures timing, transitions, focus, and end-to-end flow.
• For CLI and TUI apps, capture terminal transcripts, screenshots, recordings, generated files, and exit codes.
• For APIs and SDKs, capture requests, responses, output files, logs, and small runnable examples.
• For native apps, including Electron-type apps, prefer Agent Browser or Playwright to drive the app through its browser automation surface (Electron CDP or embedded web views) and validate acceptance criteria through normal user-visible behavior.

Walkthrough verification comes first. After walkthrough verification with Agent Browser or Playwright, create end-to-end tests that cover the new feature product area; these tests serve as the primary evidence, with walkthrough artifacts as supporting evidence. Pre-existing tests alone do not replace a walkthrough.

Supported targets

Use this skill for:

1. Web apps
2. CLI and TUI apps
3. APIs and SDKs
4. Native apps, including Electron-type apps

If the requested target is outside this list, ask whether to proceed with a best-effort evidence plan.

Hard gates

1. Scope lock first

   • Start with UAT Scope: <ticket/PR/branch scope only>.
   • If the scope is unclear, inspect the branch diff, PR, ticket, or ask the user.

2. Evidence before recommendation

   • Do not recommend merge based only on summaries, static code review, or green tests.
   • Exercise the changed behavior and save artifacts.

3. Video-first visible evidence for UI changes

   • For web, TUI, and native app work, capture video when practical, especially when the change affects interaction, layout, transitions, state changes, or error handling.
   • Also capture screenshots at meaningful checkpoints so reviewers can inspect static states.
   • If video is not practical, state why it was skipped.

4. Manual run instructions are mandatory

   • Always include steps the human can run themselves.
   • For UI work, primary instructions must use normal product-user actions.
   • For CLI, TUI, API, or SDK work, include copy-pasteable commands or code snippets with expected outcomes.

5. No GO verdict without explicit human acceptance

   • Before user confirmation, use Recommendation: Pending user sign-off.
   • Use GO or GO with follow-ups only after the user explicitly accepts.

6. Do not claim acceptance completion unilaterally

   • Evidence collection supports acceptance.
   • The human grants acceptance.

7. Adversarial evidence review

   • When subagents are available, ask at least one adversarial reviewer to compare the gathered evidence against the spec, acceptance criteria, requirements, PR description, or ticket.
   • The reviewer must provide Pass or Fail for each criterion and cite the artifact path or log line that supports the judgment.
   • If subagents are unavailable, perform the same review inline and label it Inline adversarial review.

Tool selection

Load and use the best available skill or CLI for the target, preferring Agent Browser or Playwright for UI targets. If a required skill is unavailable, install it with:

npx skills add vercel-labs/agent-browser --skill agent-browser -y
npx skills add microsoft/playwright-cli --skill playwright-cli -y

Then verify the underlying CLI required by the chosen skill.

Installation checks:

command -v agent-browser || true
command -v playwright || true

Do not install tools that are unrelated to the target.

Evidence workspace

Use the bundled helpers when practical. Resolve script paths relative to this skill directory.

# Create the evidence directory and starter manifest.
node <skill-dir>/scripts/init-evidence.mjs --target electron --scope "validated behavior"

# Run commands while saving logs, exit codes, and manifest entries.
node <skill-dir>/scripts/run-capture-command.mjs --evidence <dir> --name focused-vitest -- pnpm test

# Capture a CDP-accessible web/Electron page when agent-browser screenshots fail.
node <skill-dir>/scripts/cdp-capture-page.mjs --evidence <dir> --cdp http://127.0.0.1:9222 --title "Kata Desktop" --screenshot screenshots/start.png --text logs/start-text.txt

# Generate and check final artifacts before responding.
node <skill-dir>/scripts/write-report.mjs --evidence <dir>
node <skill-dir>/scripts/verify-evidence.mjs --evidence <dir>

The helpers do not replace judgment. They prevent common evidence mistakes: missing exit codes, empty reports, invalid evidence.json, absent UI screenshots/video, and command output that was never saved.

Write generated evidence under a repo-local folder:

uat-evidence/<target>-<YYYYMMDD-HHMMSS>/

Create artifacts as appropriate:

evidence.json          # machine-readable manifest
evidence.md            # concise human report
screenshots/           # PNG/JPG checkpoints
recordings/            # MP4/GIF terminal or UI recordings
responses/             # JSON/API/SDK outputs
logs/                  # command logs, server logs, console excerpts
outputs/               # generated files from the feature
tests/                 # end-to-end tests covering the new feature product area

Keep evidence paths stable and report them in the final answer. If uat-evidence/ is not ignored by git, mention that it should be ignored before committing unless the user explicitly wants artifacts committed.

A minimal evidence.json should include:

{
  "scope": "validated behavior",
  "target": "web|cli|tui|api|sdk|native|electron",
  "timestamp": "ISO-8601",
  "git_commit": "short sha",
  "artifacts": [
    {"type": "video|screenshot|json|log|output", "path": "relative/path", "description": "what it proves"}
  ],
  "commands": [
    {"command": "command run", "exit_code": 0, "output_path": "logs/example.log"}
  ],
  "slices": [
    {"name": "slice name", "result": "Pass|Fail", "evidence": ["relative/path"]}
  ]
}

Workflow

1. Identify scope, target, and acceptance slices

• Inspect branch diff, PR description, ticket, README, package scripts, and app entry points as needed.
• Declare UAT Scope: ....
• Declare Target: web | cli | tui | api | sdk | native | electron | mixed.
• Break the scope into 2 to 5 acceptance slices.
• Define visible pass/fail criteria for each slice.

2. Prepare the app or service

• Start the dev server, service, CLI, TUI, API, SDK fixture, or native/Electron-type app needed for the walkthrough.
• Prefer real local behavior over mocks.
• For web and native/Electron-type targets, check ports and existing processes before launch. Record any cleanup in logs/.
• Use the user's normal product command for the product smoke path. If you need CDP, traces, or deterministic screenshots, run a second instrumented path and label it as such.
• Pick free debug ports rather than assuming 9222; if a port is occupied, save the listener evidence and choose another port.
• If credentials, services, hardware, or permissions block proof, record the blocker and provide the closest reproducible plan. Do not mark blocked slices as passed.

3. Execute the feature path

Use the matching playbook:

• Web app UI flows: references/web-playbook.md
• Native/Electron-type UI flows: references/native-electron-playbook.md
• CLI, TUI, API, and SDK: references/cli-api-sdk-playbook.md

For native/Electron-type app-window validation, prefer Agent Browser (Electron CDP) or Playwright so the evidence reflects the real app surface a user sees.

For mixed work, run the user-facing path first, then the technical proof tied to the same outcome.

4. Create end-to-end tests for the product area

After walkthrough verification with Agent Browser or Playwright, create end-to-end tests that cover the new feature product area. These tests are the primary acceptance evidence.

• Use the same tool used for the walkthrough when practical: Playwright for web and Electron targets, or the repo's existing e2e framework if one is already present.
• Place tests under uat-evidence/<target>-<timestamp>/tests/ or in the repo's e2e directory when that is the project convention; record the chosen path in evidence.md.
• Each acceptance slice should map to at least one e2e test that exercises the user-visible flow, not just a unit-level assertion.
• Run the tests and save the run output (per-slice pass/fail, exit code, and trace or HTML report when available) under logs/ or tests/.
• The walkthrough proves the path; the tests lock it in as primary evidence. Do not accept a slice on tests alone without the preceding walkthrough.

5. Capture durable evidence

• Save artifacts under uat-evidence/<target>-<timestamp>/.
• Capture video for web, native/Electron-type, and TUI when practical.
• Capture screenshots at the start, key state changes, and final success state.
• Capture command output with tee, script, or scripts/run-capture-command.mjs; always save exit codes.
• Save API/SDK JSON responses and generated output files.
• Save the end-to-end test run output (per-slice pass/fail, exit code, and trace or HTML report) as primary evidence alongside walkthrough artifacts.
• For bugfix UAT, capture negative evidence that the old failure is absent: search logs for the old error string, search active source/docs for removed flags or stale commands, and save zero-match outputs.
• Classify failed checks. If a failing file or behavior changed in the branch, treat the slice as failed. If it is outside the branch diff, report it as an unrelated validation failure with file/error evidence.
• Write evidence.json and evidence.md, then run scripts/verify-evidence.mjs before responding.

6. Run adversarial evidence review

• Provide the spec, acceptance criteria, requirements, PR description, or ticket plus the saved artifacts to an adversarial review subagent when available.
• Ask the reviewer to decide Pass or Fail for each criterion and cite exact evidence paths.
• Treat missing, ambiguous, or inaccessible artifacts as failures for the affected criteria.
• If a reviewer flags a gap, collect more evidence or mark the slice failed before reporting.

7. Report results

Use this order:

1. UAT Scope: ...
2. Target: ...
3. Slice-by-slice result
4. Evidence
5. End-to-end Tests
6. Adversarial Review
7. Manual Run Instructions
8. Recommendation: Pending user sign-off
9. Please reply: accept / reject

Keep the report concise. Link to artifact paths and explain what each artifact proves.

Output contract

For every UAT response, include the sections below. Include the End-to-end Tests section when the walkthrough used Agent Browser or Playwright (web and native/Electron-type targets).

UAT Scope: <scope>
Target: <web|cli|tui|api|sdk|native|electron|mixed>

Slice-by-slice result:
- Pass/Fail: <slice> - <one-line evidence summary>

Evidence:
- <artifact path> - <what it proves>

End-to-end Tests:
- <test path> - <slice it covers> - Pass/Fail (run: <log/report path>)

Adversarial Review:
- Pass/Fail: <criterion> - <artifact-backed reason>

Manual Run Instructions:
1. <human step or command>
   Expected: <visible result or output>

Recommendation: Pending user sign-off
Please reply: accept / reject

For UI targets, manual instructions must start with normal user actions in the running app. Put automation commands or test harnesses only in a clearly labeled Fallback (Engineering Only) section.

For CLI, TUI, API, and SDK targets, manual instructions may be commands or code snippets, but they must include expected output and any required environment variables.

Ticket update guidance

If a ticket or PR is known, offer or perform a status update with:

• Scope validated
• Target and tools used
• Slice pass/fail summary
• Evidence paths or links
• Manual run instructions
• Current recommendation

Before explicit user acceptance, label the update Pending user sign-off. After explicit acceptance, record the final verdict if the project workflow requires it.

Pre-response self-check

Before responding, verify:

• Scope is stated.
• Target is stated.
• Each slice has pass/fail status.
• Evidence artifacts exist or blockers are clearly labeled.
• evidence.json is valid and evidence.md is non-empty.
• UI evidence includes video plus screenshots when practical, or a stated reason video was skipped.
• UI screenshots/video files exist and are readable.
• Native/Electron-type app validation used Agent Browser (Electron CDP) or Playwright, or explains why another tool was used.
• End-to-end tests covering the new feature product area were created and run, with per-slice pass/fail and run output saved as primary evidence (UI targets).
• Adversarial review checked every criterion against evidence and produced pass/fail judgments.
• Command logs include exit codes.
• Old-bug negative evidence is saved when validating a fix.
• Dev servers, native/Electron-type apps, or background processes launched for UAT are cleaned up or explicitly left running for the user.
• Manual run instructions are included.
• Recommendation remains Pending user sign-off unless the user has accepted.

Common mistakes

• Running only pre-existing tests without a walkthrough or newly created end-to-end tests covering the product area.
• Reporting a code summary without artifacts.
• Skipping video or screenshots for UI work without explanation.
• Providing Playwright or test commands as the primary manual UI path.
• Forgetting TUI visual evidence.
• Omitting API request and response files.
• Declaring GO before the user accepts.
• Installing every tool instead of selecting the best tool for the target.