Create a verification skill
Every serious project needs a scripted way to drive the real app and prove behavior: launch it, exercise a feature the way a user would, and capture evidence. This skill generates that as a project-local skill (.pi/skills/verify-<app>/) tailored to the repo. You write the generator's output for the next agent, not for a human: it will be read cold, mid-task, by an agent that has never seen the app.
1. Interview the repo, not the user
Answer these from the codebase and only ask the user what you cannot observe:
- Surface: what does a user actually touch? A web UI, a CLI/TUI, a desktop app, an API, a mobile app, a library? A repo can have several; pick the primary one and note the rest.
- Run: how does the app start locally? Prefer the repo's own documented dev command (package scripts, Makefile, README quickstart). Note ports, env vars, seed data, auth.
- Drive: how can an agent interact with it programmatically? Existing harnesses come first. Look for Playwright/Cypress specs, expect scripts, PTY helpers, curl-able endpoints, or a debug port. Only then pick a generic recipe. Use browser/CDP for web and Electron, a tmux/PTY harness for CLI/TUI, or plain HTTP for services.
- Observe: what evidence can be captured? Screenshots, terminal transcripts, response bodies, logs, exit codes, DB state.
- Isolate: can two instances run side by side (ports, data dirs, profiles)? If not, say so in the generated skill: refusing to double-drive a shared instance beats corrupting the user's session.
If the checkout doesn't build or start as-is, fix that first (or report it precisely) before generating; a skill written against a broken base teaches wrong steps. When an irrelevant missing asset blocks startup (a static dir the API never serves, a sample config), the generated skill may create it, clearly marked as verification scaffolding, and remove it in cleanup.
2. Generate the skill
Write .pi/skills/verify-<app>/SKILL.md with YAML frontmatter. Set name: verify-<app> and add a description that names the app, the surface, and when to reach for it. Without frontmatter, the skill never registers. Include these sections, ground each in what the interview actually found, and leave no placeholders:
- Launch: the exact command that starts the app for verification, and how to tell it's ready (a log line, a port answering, a prompt). Include teardown. For a short-lived CLI or TUI there is no server to keep alive: launch means build the binary (or install deps) once, then start each drive in its own isolated PTY or tmux session.
- Doctor: one read-only check that answers "is this instance worth driving?" It confirms the process is up, the version or build is right, the port is owned by us, and auth is valid. An agent runs this first whenever anything looks off.
- Drive: the harness recipe with real selectors/commands from this repo, not examples. Prefer stable handles (ARIA labels, data attributes, prompt strings, route paths) over coordinates and tab order.
- Evidence: what to capture for a proof and where it goes. State the proof standards: exercise the real user path, not internal setters or test-only endpoints; capture the action and the resulting state, not just the final screen; verify side effects (files written, rows inserted, messages sent) alongside what's visible; mocks only where a production boundary already isolates the external system. When the safe path is a dry-run or test mode, verify what it actually skips by observing (files, network, git refs) rather than trusting its name: some dry-runs still touch the network or open a browser.
- Cleanup: how to tear down instances the run created. Never kill by process name; kill what you started. Cleanup removes instances and scratch state, never the evidence: proof artifacts survive the teardown, in a location the skill names.
- Helpers: any script the skill ships is executable and its invocation is shown in the skill body. A helper the reader has to reverse-engineer is not a helper.
3. Seed the feature map
Create .pi/skills/verify-<app>/features/README.md plus one file per user-facing feature you can identify (aim for the top 3-5 to start, from routes, commands, menus, or docs). Follow the shape in references/feature-map-example/, with a README index and one file per feature. Each file answers, from the user's point of view: what the feature is, how to reach it, how to drive it with the harness, and what observable end state proves it works. The four H2s are Sub-features, How to get to it (user POV), Driving it with <harness>, and Gotchas. The map is the repo's maintained verification source; a proof that drives one convenient entry point is incomplete when the map lists others.
4. Prove the generated skill before handing it over
Run its own instructions end to end once. Launch it, run doctor, drive ONE mapped feature, capture evidence, and clean up. One feature is enough because the map lets later runs cover the rest. After cleanup, confirm the evidence still exists at the named location. A cleanup that eats the proof fails this step. Fix what fails. Run the generated cleanup after every failed iteration so broken attempts don't strand processes and ports. A generated skill that was never executed is a draft, not a deliverable.
5. Offer the maintenance loop
Point the user at /skill:maintain-verification-skill for keeping the map honest as the app changes. Suggest a cadence only if they ask.
1---2name: create-verification-skill3description: Generate a project-local verification skill that drives your app the way a user does in any language, framework, or platform. Use for /skill:create-verification-skill, "make a control skill for this repo", or when a project has no scripted way to prove UI/CLI/service behavior.4---56# Create a verification skill78Every serious project needs a scripted way to drive the real app and prove behavior: launch it, exercise a feature the way a user would, and capture evidence. This skill generates that as a project-local skill (`.pi/skills/verify-<app>/`) tailored to the repo. You write the generator's output for the next agent, not for a human: it will be read cold, mid-task, by an agent that has never seen the app.910## 1. Interview the repo, not the user1112Answer these from the codebase and only ask the user what you cannot observe:1314- **Surface:** what does a user actually touch? A web UI, a CLI/TUI, a desktop app, an API, a mobile app, a library? A repo can have several; pick the primary one and note the rest.15- **Run:** how does the app start locally? Prefer the repo's own documented dev command (package scripts, Makefile, README quickstart). Note ports, env vars, seed data, auth.16- **Drive:** how can an agent interact with it programmatically? Existing harnesses come first. Look for Playwright/Cypress specs, expect scripts, PTY helpers, curl-able endpoints, or a debug port. Only then pick a generic recipe. Use browser/CDP for web and Electron, a tmux/PTY harness for CLI/TUI, or plain HTTP for services.17- **Observe:** what evidence can be captured? Screenshots, terminal transcripts, response bodies, logs, exit codes, DB state.18- **Isolate:** can two instances run side by side (ports, data dirs, profiles)? If not, say so in the generated skill: refusing to double-drive a shared instance beats corrupting the user's session.1920If the checkout doesn't build or start as-is, fix that first (or report it precisely) before generating; a skill written against a broken base teaches wrong steps. When an irrelevant missing asset blocks startup (a static dir the API never serves, a sample config), the generated skill may create it, clearly marked as verification scaffolding, and remove it in cleanup.2122## 2. Generate the skill2324Write `.pi/skills/verify-<app>/SKILL.md` with YAML frontmatter. Set `name: verify-<app>` and add a `description` that names the app, the surface, and when to reach for it. Without frontmatter, the skill never registers. Include these sections, ground each in what the interview actually found, and leave no placeholders:2526- **Launch:** the exact command that starts the app for verification, and how to tell it's ready (a log line, a port answering, a prompt). Include teardown. For a short-lived CLI or TUI there is no server to keep alive: launch means build the binary (or install deps) once, then start each drive in its own isolated PTY or tmux session.27- **Doctor:** one read-only check that answers "is this instance worth driving?" It confirms the process is up, the version or build is right, the port is owned by us, and auth is valid. An agent runs this first whenever anything looks off.28- **Drive:** the harness recipe with real selectors/commands from this repo, not examples. Prefer stable handles (ARIA labels, data attributes, prompt strings, route paths) over coordinates and tab order.29- **Evidence:** what to capture for a proof and where it goes. State the proof standards: exercise the real user path, not internal setters or test-only endpoints; capture the action and the resulting state, not just the final screen; verify side effects (files written, rows inserted, messages sent) alongside what's visible; mocks only where a production boundary already isolates the external system. When the safe path is a dry-run or test mode, verify what it actually skips by observing (files, network, git refs) rather than trusting its name: some dry-runs still touch the network or open a browser.30- **Cleanup:** how to tear down instances the run created. Never kill by process name; kill what you started. Cleanup removes instances and scratch state, never the evidence: proof artifacts survive the teardown, in a location the skill names.31- **Helpers:** any script the skill ships is executable and its invocation is shown in the skill body. A helper the reader has to reverse-engineer is not a helper.3233## 3. Seed the feature map3435Create `.pi/skills/verify-<app>/features/README.md` plus one file per user-facing feature you can identify (aim for the top 3-5 to start, from routes, commands, menus, or docs). Follow the shape in [`references/feature-map-example/`](references/feature-map-example/), with a README index and one file per feature. Each file answers, from the user's point of view: what the feature is, how to reach it, how to drive it with the harness, and what observable end state proves it works. The four H2s are `Sub-features`, `How to get to it (user POV)`, `Driving it with <harness>`, and `Gotchas`. The map is the repo's maintained verification source; a proof that drives one convenient entry point is incomplete when the map lists others.3637## 4. Prove the generated skill before handing it over3839Run its own instructions end to end once. Launch it, run doctor, drive ONE mapped feature, capture evidence, and clean up. One feature is enough because the map lets later runs cover the rest. After cleanup, confirm the evidence still exists at the named location. A cleanup that eats the proof fails this step. Fix what fails. Run the generated cleanup after every failed iteration so broken attempts don't strand processes and ports. A generated skill that was never executed is a draft, not a deliverable.4041## 5. Offer the maintenance loop4243Point the user at `/skill:maintain-verification-skill` for keeping the map honest as the app changes. Suggest a cadence only if they ask.
shishiv