Verify

Verify

Overview

/verify is the final gate before a change ships. Unit tests prove the code compiles and the methods return what their authors expected. They do not prove the running service accepts real payloads, returns the documented envelope, preserves cross-service contracts, or survives the weird shapes production traffic actually produces. /verify does that.

Core principle: No completion claim is trustworthy until a curl against a running service produces the right response body, status code, and side effects — with fresh, recorded output. Everything else is paraphrase.

Design goals every run must optimize for:

• Evidence, not assertion — every "pass" is backed by recorded curl -i output stored under the ticket folder
• Payload realism — fixtures come from real row shapes (DB sample) or user-provided cases, not invented JSON
• Variant coverage — happy path + at least one boundary + at least one negative per endpoint
• Iteration discipline — on failure, pass a specific delta to /work, not "it broke, fix it"
• Escalation bounds — unproductive /work loops are detected and escalated, not retried forever
• Environment-bounded blast radius — local/dev is default; audit/stg payload runs are allowed only when the user explicitly names that target; prod is never allowed

The Iron Law

NO "PRODUCTION READY" CLAIM UNTIL EVERY ASSERTION PASSES AGAINST A LIVE TARGET SERVICE
NO LIVE VERIFICATION OUTSIDE THE USER-AUTHORIZED TARGET AND DATA SCOPE
NO API VERIFICATION WITHOUT A GENERATED CURL HARNESS, RECORDED `curl -i` OUTPUT, AND SAVED REQUEST FIXTURES
NO REACHABLE HTTP ENDPOINT MAY BE VERIFIED BY UNIT TESTS, BYTECODE, MAPPER SQL, OR CODE READING INSTEAD OF CURL
NO PAYLOAD RUN AGAINST prod FROM THIS SKILL
AUDIT/STG PAYLOAD RUNS REQUIRE EXPLICIT USER INSTRUCTION AND RECORDED TARGET ENVIRONMENT
NO SKILL EXIT UNTIL verify.md IS WRITTEN AND ITS STATUS IS  pass | fail-escalated | fail-user-required

When to Use

Normal mode — the pipeline position:

/explore  →  explore.md
/work     →  work.md   (code + unit/integration tests + lint/build)
/verify   →  verify.md (HTTP-level E2E proof; re-triggers /work on failure)

Invoke /verify when any holds:

1. /work has just completed for <YYYYMM>/<slug> and the user wants proof before merging.
2. A paired work.md exists but the current session has no fresh HTTP-level evidence.
3. The user supplies an API path + at least one payload and asks for ad-hoc verification (standalone mode — see below).
4. A bug report arrived against an existing endpoint and you want to reproduce + gate the fix before touching code. (In this case /verify runs first, captures the failing curl, then hands off to /explore → /work → /verify.)

Skip only for:

• Pure frontend-only diffs with no backend surface touched (run npm run dev and use the browser; a separate skill owns frontend QA).
• Config-only changes that do not alter any HTTP contract.
• Doc/comment-only commits.

Standalone Mode (no ticket folder)

If there is no <artifact-root>/.backend/<YYYYMM>/<slug>/ folder and the user wants ad-hoc API verification:

1. Identify endpoint (method + path), target service, and sample payload from the request or existing fixtures; ask only for missing inputs or data-access authorization.
2. Derive a slug from the endpoint (e.g. verify-post-users-login) and today's <YYYYMM>.
3. Choose the artifact root:
   • If the endpoint belongs to a single microservice/module, use that service directory. Example: example-viewer-api/.backend/<YYYYMM>/<slug>/.
   • If the endpoint spans multiple services or is explicitly repo-wide, use the repo root.
   • If unclear, infer from controller/service paths; ask only when multiple service roots are equally plausible.
4. Create <artifact-root>/.backend/<YYYYMM>/<slug>/ and proceed from Phase 2 onward. explore.md and work.md will not exist — record that in Phase 0 instead of loading them.
5. On Phase 6 write only verify.md (no docs entry — that is /work's job).

Inputs & Outputs

Inputs

Outputs (all three are required for completion)

Folder layout (per ticket, post-verify):

<artifact-root>/.backend/
  <YYYYMM>/
    <slug>/
      explore.md          ← from /explore
      work.md             ← from /work
      verify.md           ← this skill
      harness/
        <endpoint>.sh     ← one curl harness per endpoint (executable)
        _env.sh           ← BASE_URL / TOKEN / COMMON_HEADERS
      fixtures/
        <endpoint>.<variant>.json
        ...
      runs/
        run-1.log         ← full curl -i output for iteration 1
        run-2.log         ← (only exists if /work was re-invoked)
        run-3.log

See verify-template.md for the exact verify.md format and curl-harness.md for the shell script template.

Execution Rules

Language: Match the user's language for user-facing responses, verification plans, final summaries, and docs notes. Keep code symbols, file paths, SQL, endpoints, commands, and exact error strings unchanged.

Required:

• In Phase 0, state the target and verification plan. Reuse existing authorization; ask only for missing target/data permissions.
• Prepare harnesses and inspect code within scope. Before live calls or data sampling, verify the target and side effects are authorized; do not run unapproved shared-environment mutations.
• Perform verification directly; delegate bounded independent cases only when useful, available, and authorized.
• Verifier subagents write only under the ticket folder: harness/*.sh, fixtures/*.json, runs/run-<N>.log.
• The main agent writes the approved plan, verifier work log, and result matrix to <artifact-root>/.backend/<YYYYMM>/<slug>/verify.md.
• If a /work docs entry exists, append only a Verification (/verify) section to that file. In standalone mode, do not create a new docs entry.

Plan must include: targets, env/base URL, payload sources, variants, assertions, artifact paths, safety bounds, subagent assignments.

Forbidden:

• prod payload run.
• audit/stg run without explicit user instruction.
• Substituting unit tests or code reading for a reachable endpoint.
• Delegating without a bounded task, available tools, and authorization.

Verifier handoff: scope, artifacts, commands, result matrix rows, failures/blockers, redaction notes.

Quality gates:

• Done = approved plan recorded, fixtures/harness/run logs exist, verify.md §6 matrix matches latest runs/run-<N>.log, status is valid.
• Scope = writes only under ticket .backend/<YYYYMM>/<slug>/{harness,fixtures,runs,verify.md} plus optional docs-note append.
• Evidence = HTTP claim cites runs/run-<N>.log:line; payload claim cites fixture/source; DB claim records read-only query shape + redaction note.
• Failure = FAIL-APP service behavior wrong, FAIL-CONTRACT explore/work contract wrong, FAIL-ENV target/auth/downstream unavailable, FAIL-DATA payload source invalid, PASS-WITH-NOTES non-blocking gap.
• Escalate = FAIL-APP creates bounded /work delta with user approval; FAIL-CONTRACT returns to /explore; FAIL-ENV/FAIL-DATA asks user for env/data.

Final response: Report the absolute verify.md path, status, endpoint/variant pass-fail, work performed and, if delegated, verifier ownership, and blockers, then STOP.

The Seven Phases

Complete each phase before the next. Do not interleave. The loop between Phase 4 and Phase 5 is the only legal loop; no ad-hoc retries.

Phase 0 — Load & Validate the Contract

1. Determine artifact root and ticket folder — ask or infer <YYYYMM>/<slug>. Read explore.md and work.md in full if they exist.
   • If work.md §4 or the user scope points to a single microservice/module, use that service directory as artifact root.
   • If the change spans multiple services or is explicitly repo-wide, use the repo root.
2. Announce the kind (Bug / Feature / Refactor), the scope one-liner from explore.md §2, and whether /work reported status: done | partial | blocked in work.md §10 re-entry header.
3. From work.md §4 Changes and explore.md §7 Proposed Changes, derive the endpoint target list — the minimal set of HTTP surfaces that must be probed:
   • Controllers / @RequestMapping that were edited or added.
   • Feign client methods that were added/extended (these imply an endpoint on the callee service that now has a new contract).
   • Endpoints exercised by any test that was added/edited in work.md §5.
4. For each target, record: service, method, path, auth required?, content-type, sample response envelope citation (file:line).
5. Validate:
   • At least one endpoint target was derivable. If zero targets, stop — the change had no HTTP surface and /verify is the wrong skill.
   • Each target's service exposes a reachable local/dev profile, or the user explicitly asked to verify an audit/stg target. Confirm ports/profile from the service's CLAUDE.md, profile config, process command, or target URL.
   • work.md §10 status is not blocked. If blocked, stop and tell the user /work must complete before /verify runs.
6. Confirm the target environment:
   • Default: local/dev service on localhost, 127.0.0.1, 0.0.0.0, or host.docker.internal.
   • Allowed by explicit user instruction: audit or stg, including a locally bound instance running that profile.
   • Never allowed: prod or any hostname/base URL that resolves to production.
7. Record the target env, BASE_URL, and evidence that the user asked for audit/stg if not local/dev. If the target is ambiguous, stop and ask.

Output of this phase: a verification plan checkpoint, printed in chat, with:

• target env and proof it is allowed
• endpoint target list
• payload/variant plan
• harness/run-log artifact plan
• assertion plan
• subagent assignment plan
• existing authorization or the specific missing permission

Continue the authorized checks. Pause only dependent live actions when the environment, account, endpoint, or mutation scope needs a decision; preparation does not require another approval.

Phase 1 — Payload Sourcing

Follow payload-sourcing.md in this directory. Priority order, top wins:

1. User-provided payloads — explicitly pasted in the conversation or attached to the ticket. Save each under fixtures/<endpoint>.user-<label>.json verbatim (redact obvious PII before writing).
2. Saved fixtures — any <artifact-root>/.backend/<YYYYMM>/<slug>/fixtures/*.json from a prior /verify run on this ticket. Reuse when the endpoint contract has not changed since (compare explore.md §5B contract with the fixture shape).
3. Live MySQL sampling — only if the prior two are insufficient. Read-only, dev-only, LIMIT-bounded. Shapes are copied; PII is redacted to placeholders (__REDACTED_EMAIL__, __REDACTED_PHONE__). This procedure lives in payload-sourcing.md.
4. Synthesized payloads — last resort. Only for boundary/negative variants where no real row would look that way (e.g. empty array, max-length string, invalid enum value). Every synthesized field must be justified in verify.md §4.

For each endpoint target, produce at minimum three variants:

Bug fixes get one additional mandatory variant:

Write each variant to fixtures/<endpoint>.<variant>.json. For GET/DELETE requests without a JSON body, the fixture still records the full request shape: method, path, query parameters, path variables, headers that affect behavior, and expected assertions. Never commit raw PII — apply redaction before write.

Record fixture paths, source category, redaction decisions, and any sample-query shape used. A delegated verifier returns the same evidence.

Phase 2 — Harness Generation

Follow curl-harness.md in this directory. For each endpoint target, generate harness/<endpoint>.sh with this shape:

• Loads _env.sh which exports BASE_URL, TOKEN (if needed), and COMMON_HEADERS.
• One run_variant() invocation per variant fixture.
• curl -i --max-time 10 -s -S -o >(tee -a ../runs/run-<N>.log) — always includes -i to capture status + headers, always bounded by timeout.
• Captures: status code, response body, elapsed time. Does not follow redirects unless the endpoint documents redirect behavior.
• Exits non-zero if any variant failed its declared assertion.

Declare assertions per variant inline in the shell script:

assert_status 200
assert_json_path '.data.userId' '<expected>'
assert_json_key_present '.data.createdAt'
assert_header 'Content-Type' 'application/json'

Assertion helpers are defined in harness/_assert.sh (generated once per ticket). Keep them pure bash + jq + grep. No Python, no Node — the harness must run in any local dev environment with a shell and jq installed.

Auth strategy per service family:

• Services behind an auth filter → obtain a dev credential from the project's auth authority in the dev profile, such as GET /test/generate-jwt, for local/dev runs. For shared environments, use only the user-provided or already available environment token/session for that target; never mint or paste a prod token. The same token is reusable across other services when they share the same issuer or signing key. Full reference: jwt-auth-reference.md in this directory — contains stack-neutral auth schemes plus ready-to-adapt _env.sh and _preflight.sh snippets.
• Feign-only internal endpoints (no external auth): still send the standard X-Request-Id header.
• Services with no auth in the target profile: skip the token step but assert the profile/target really matches the recorded TARGET_ENV when exposed (e.g. actuator/env/actuator/info, process command, or service banner).
• Services with a different auth scheme must use their own section of jwt-auth-reference.md; do not assume JWT applies everywhere.

Cache the token in _env.sh for the run; never paste it into fixture files. Tokens have exp — re-run _preflight.sh on every iteration (see curl-harness.md §Re-Run Protocol).

Do not generate harness files for endpoints you cannot reach in the selected target. If the target service cannot start or is unreachable (missing downstream, auth unavailable, wrong profile), stop, record it in verify.md §9 as [BLOCK], and ask the user.

If delegated, each verifier owns only its assigned harness files.

Phase 3 — Execution

1. Start the target service(s) if not already running:
   • cd <service> && ./gradlew bootRun --args='--spring.profiles.active=dev' (in background or a separate terminal the user is already running), unless the user explicitly directed an already-running audit/stg instance.
   • Poll a health endpoint (/actuator/health or the service's documented readiness probe) until 200 OK or until a 60s cap. Do not start timing tests before ready.
2. Execute each harness/<endpoint>.sh. Each run appends to runs/run-<N>.log where <N> is the current iteration (starts at 1; increments each time Phase 5 re-invokes /work).
3. Capture every request/response pair verbatim. curl -i headers + full body. Redact the Authorization: header value in the log before committing.
   • If the endpoint is reachable, keep running the curl variants until happy, boundary, and negative payloads have all produced recorded HTTP results or a concrete app/contract failure is found.
   • Unit tests, bytecode inspection, mapper SQL checks, and code reading are supplemental evidence only. They may explain or narrow a failure, but they do not satisfy /verify for any reachable HTTP endpoint.
   • If auth/session/downstream state prevents a real happy-path curl, classify it as FAIL-ENV or fail-user-required; do not mark the endpoint as passing based on non-HTTP evidence.
4. Compute the variant result matrix:

endpoint          variant     status  latency(ms)  assertions  result
POST /users/login happy       200     142          3/3         ✓
POST /users/login boundary    400     89           2/2         ✓  (expected 400)
POST /users/login negative    401     91           2/2         ✓
POST /users/login regression  500     154          1/3         ✗

5. Never paraphrase the result matrix. It is copied verbatim into verify.md §5.

If any service fails to start or a probe times out, record the actual error and stop — do not retry silently.

Record exact commands, run-log paths, matrix rows, and pass/fail classifications. Review any delegated evidence before incorporating it; missing evidence cannot become a passing claim.

Phase 4 — Triage

Classify the matrix:

For FAIL-APP and FAIL-CONTRACT, prepare a rework delta — a structured handoff to /work:

endpoint:       <method> <path>
variant:        <variant-name>
request:        <fixture path>
expected:       <status>, body path <path> == <value>, header <name>=<value>
observed:       <status>, body <snippet>, header <name>=<value>
trace-hint:     <service log excerpt or absence thereof>
likely-source:  <file:line from work.md §4 that owns this behavior, if derivable>

This delta is the input to Phase 5. It is also saved into verify.md §7 for the run log.

Phase 5 — Iteration (bounded)

Iteration cap: 3 /work re-invocations per /verify session.

For each fail classification:

1. State the rework delta. If fixing the failure is already authorized, proceed within that scope; for verification-only requests, ask before changing source.

2. With existing or new authorization: invoke /work with explicit scope limited to the delta. Pass the delta as the opening message and point /work at the same <YYYYMM>/<slug> folder. /work appends a new entry to its work.md §2 Contract Adherence ("rows added mid-flight via /verify rework delta #N"). Do not let /work widen scope beyond the delta.

3. On user reject / hold: stop. Write verify.md with status: fail-user-required and the delta as the question.

4. After /work returns, re-run Phase 3 (execution) with the same harness — do not regenerate fixtures unless the contract itself changed. Append to runs/run-<N+1>.log.

5. Compare iteration N+1 to iteration N. Meaningful progress = at least one of:

   • A previously failing variant now passes.
   • The failing variant moved to a different, more specific failure (e.g. 500 → 400 with a validation message — indicates the service is now routing the request correctly but rejecting its content).
   • The rework delta's likely-source line was edited and the corresponding assertion's observed value changed (even if still wrong).

   If none of these hold, the iteration did not make meaningful progress.

6. Escalation ladder:

iteration 1 fails           → generate scoped rework delta; follow Phase 5 authorization
iteration 2 fails AND no progress vs iter 1
                            → STOP the /work loop.
                              Ask user: (a) re-run /explore (contract may be wrong),
                                        (b) accept verified scope + defer, or
                                        (c) provide new evidence / payloads.
iteration 2 fails WITH progress
                            → one more /work attempt allowed (iteration 3).
iteration 3 fails (any)     → STOP. status: fail-escalated.
                              Write verify.md §9 [BLOCK] with the specific decision the user must make.
                              Do NOT re-invoke /work.

7. If at any iteration the failure classification flips to FAIL-CONTRACT, skip the remaining /work budget and escalate to /explore immediately — the contract document is the root cause, not the code.

Phase 6 — Wrap Up (verify.md)

Write <artifact-root>/.backend/<YYYYMM>/<slug>/verify.md using verify-template.md in this directory. Required sections:

1. §1 One-paragraph recap (prose — the only prose section, plain language primary).
2. §2 Meta (kind/slug/yyyymm/status/iteration-count).
3. §3 Verification Plan & Agent Work Log (approved plan + verifier subagent handoffs).
4. §4 Endpoint Target List (matches Phase 0 output).
5. §5 Payload Sources (origin of each fixture — user / saved / DB-sampled / synthesized, with redaction notes).
6. §6 Results Matrix (verbatim from the last iteration's execution).
7. §7 Cross-Service Checks (Feign consumers, SSE channels, Kafka payloads — what was probed and the result).
8. §8 Iteration Trail (one row per /work re-invocation with the delta and the delta's effect).
9. §9 Performance Observations (p95 latency per endpoint, if meaningfully measured).
10. §10 Open Items ([BLOCK] / [INFO] / [GAP]).
11. §11 Re-entry Header (how a future session reloads context).

Also:

• Each harness/<endpoint>.sh and each fixtures/*.json remains in the ticket folder — they are reusable evidence.
• runs/run-<N>.log files stay. They are the audit trail. Do not prune between iterations.
• Append a "Verification (/verify)" section to the existing <artifact-root>/docs/features/<YYYY-MM-DD>-<slug>.md or <artifact-root>/docs/bugs/...md change note written by /work, summarizing the verification in 3-5 lines of plain language. If the docs entry does not exist (standalone mode), do not create one.

After writing, print:

• Absolute path to verify.md
• Final status: pass | fail-escalated | fail-user-required
• A 3-5 line human summary of: endpoints probed, variants run, pass/fail counts, what (if anything) blocks merging.
• A clear explanation of what the main agent did and what each verifier subagent did.

Then STOP. Do not commit, push, or open a PR. /verify, like /work, ends at the filesystem.

Scope Boundaries

Integration with Other Skills

Sibling skills in this pipeline:

• work (typical predecessor) — produces work.md whose §4 / §5 / §7 define the endpoint target list and cross-service checks. /verify reads but never edits work.md.
• explore (rerun target on FAIL-CONTRACT) — if the contract is wrong, escalate here rather than patching in code.

Support files in this directory (always used):

• payload-sourcing.md — Phase 1 payload sourcing procedure with PII redaction
• curl-harness.md — Phase 2 harness template (_env.sh, _assert.sh, per-endpoint scripts) + Re-Run Protocol
• jwt-auth-reference.md — auth acquisition procedure for Phase 2 (currently written for one specific codebase's JWT scheme — adapt to your stack's auth flow)
• verify-template.md — Phase 6 verify.md output format

Optional companion skills (if also installed — e.g. superpowers):

This skill is self-contained — the plan gate, fresh-output rule, and verifier-subagent dispatch are inlined. The following are optional enhancers if present in your agent's skill registry:

• verification-before-completion — alternative formulation of the "fresh output only" rule; applies at unit/integration layers where /verify does not operate
• systematic-debugging — use when a FAIL-APP triage cannot identify a plausible likely-source; do not hand /work a rework delta without a call-chain hypothesis
• test-driven-development — not invoked directly by /verify, but any regression variant added here becomes a candidate test for the next /work pass
• dispatching-parallel-agents — alternative formulation of the parallel-endpoint dispatch technique for Phase 3 with 3+ independent endpoints
• finishing-a-development-branch — use AFTER /verify exits with status: pass, for commit / PR ceremony
• jira-update / jira-ascli — use AFTER /verify exits, for ticket status updates with verification evidence

Quick Reference

Citations & Redaction Discipline

• Every claim about the service's behavior must cite the exact runs/run-<N>.log:line range.
• Every claim about the contract must cite explore.md §5B or the controller/DTO file:line.
• PII categories to redact before writing any file under .backend/: email addresses (replace with user+<id>@example.test), phone numbers, national ID, real student names (replace with student<id>), real school names (replace with school<id>).
• Authorization: header values in run logs → replace with Bearer __REDACTED__.
• Cookies / session IDs → replace value with __REDACTED__.
• If a field you cannot classify appears in a sampled row, redact it to __UNKNOWN__ and note the field in verify.md §4 — do not commit it.