Feature Verify — Runtime-First API Verification
Verify deployed behavior with read-only runtime evidence:
Primary-model analysis → optional independent reviewer → integrated verdict
Use this for post-deploy checks, smoke tests, and production diagnosis. For
local tests use the repository verification route; for changes use
flow-drive; for code review use
change-verdict.
When NOT to Use
- The requested action changes application data, configuration, or deployment state.
- The task is to implement or modify a feature; use
flow-drive.
- The task is a code, security, or test review rather than runtime behavior
verification.
- There is no read-only endpoint, observation path, or approved test environment that
can produce evidence for the requested claim.
Required References
Read these at the indicated phase:
- P0/P3: safety-rules.md and environments.md
- P1/P2/P4/P5: blackbox-testing.md
- P0/P2/P5: verification-playbook.md
- Report: output-template.md
Bash is permitted only for read-only curl and observation queries reviewed against the safety rules. Missing endpoint allowlist means P3 is skipped.
P0 — Scope and Safety
- Select
--env, defaulting to test, and load its configuration.
- Run
scripts/health-probe.sh <health-url> (3 attempts, 2-second timeout).
- Compare local HEAD with the deployed version and warn on mismatch.
- Confirm every planned operation is read-only.
- Determine L4/L3/L2-API/L2-OBS/L1 using the playbook matrix.
API unreachable with logs gives L2-OBS; without logs gives L1. At L1 skip P3/P4. --level L2 remains an alias for L2-API.
P1 — Diff-Lite Scope
Follow blackbox-testing.md § P1. Map git diff main...HEAD --name-only to affected endpoints, dependency chains, active triggers, and passive targets. This phase scopes behavior; it does not judge code quality. If no diff exists, build scope from the user's feature description.
P2 — Test Charter and Approval
Follow blackbox-testing.md § P2. Generate only cases supported by the detected level: L1 regression, L2 active trigger, L3 passive observation, and M1 metrics. Present the charter and wait for user approval before any P3 request.
P3 — API Execution
Run only for L2-API/L3/L4 after approval. For each allowlisted case:
- Confirm method and endpoint exactly match the allowlist.
- Run one request at a time with fixed, non-PII test parameters:
scripts/api-exec.sh <GET|POST> <allowlisted-url> [json-payload]
- Record its request ID, HTTP status, response fields, and latency.
The script rejects methods other than GET and POST; the operator must separately confirm that POST is allowlisted and read-only. L2-OBS always skips P3.
P4 — Observation Correlation
Follow blackbox-testing.md § P4.
- L3/L4: correlate each request ID, then fall back to alternate fields and endpoint/time window.
- L2-OBS/L3/L4: scan the observation window for related errors and warnings.
- Observe background services after their expected delay; query affected metrics at L4.
- Record blind spots. A missing log signal is not itself proof of failure.
For L2-OBS use deploy time → now, a user-provided window, or the last 30 minutes, in that order.
P5 — Independent Verdict
- The primary model forms a conclusion from P3/P4 evidence.
- Only when the caller explicitly requests a second opinion, dispatch a fresh,
isolated read-only reviewer with P1/P3/P4 evidence. The reviewer must assess
scope, commands, allowlist compliance, evidence, blind spots, and confidence
without receiving the primary conclusion. An explicit
--second-opinion=codex-exec may supply an additional one-shot CLI review;
it is additive and opt-in, with no hidden fallback.
- If no second opinion is requested or available, label the result
primary model only and state plainly: "Only the primary
model's verdict is present; no independent review ran." Do not describe the
primary result as independently verified.
- Integrate available conclusions using the verdict/confidence rules in the
playbook, keeping a skipped or unavailable reviewer visible.
- Render output-template.md.
The report may recommend another skill, but must not auto-invoke it.
Output
Produce the report from references/output-template.md. It must identify the deployed
target and local/deployed alignment, record the detected degradation level, list every
executed and skipped case with the reason, preserve request/log/metric evidence, state
blind spots, and end with the integrated verdict and confidence. A skipped API phase is
valid only when the level or safety gate requires it; it is not a passing execution.
Verification
1---2name: feature-verify3disable-model-invocation: true4description: 'Feature verification (READ-ONLY, P0-P5). Purpose: verify feature behavior after deployment, validate API responses, diagnose production issues, or run a post-deploy smoke test. Not for: modifying data (use flow-drive), code review (use change-verdict), writing tests (use tdd-workflow), security audit (use change-verdict). Output: a P0-P5 verification report with pass/fail evidence per check.'5allowed-tools: 'Read, Grep, Glob, Bash, Bash(codex:*), WebFetch, Agent, Skill'6context: fork7metadata:8 source: https://skillmd.com/skills/hmj1026/feature-verify9 dhpk-name-style: portable-skill10 dhpk-capability-id: dhpk.skill.feature-verify11 dhpk-invocation-class: explicit-only12---1314# Feature Verify — Runtime-First API Verification1516Verify deployed behavior with read-only runtime evidence:1718`Primary-model analysis → optional independent reviewer → integrated verdict`1920Use this for post-deploy checks, smoke tests, and production diagnosis. For21local tests use the repository verification route; for changes use22`flow-drive`; for code review use23`change-verdict`.2425## When NOT to Use2627- The requested action changes application data, configuration, or deployment state.28- The task is to implement or modify a feature; use29 `flow-drive`.30- The task is a code, security, or test review rather than runtime behavior31 verification.32- There is no read-only endpoint, observation path, or approved test environment that33 can produce evidence for the requested claim.3435## Required References3637Read these at the indicated phase:3839- P0/P3: [safety-rules.md](references/safety-rules.md) and [environments.md](references/environments.md)40- P1/P2/P4/P5: [blackbox-testing.md](references/blackbox-testing.md)41- P0/P2/P5: [verification-playbook.md](references/verification-playbook.md)42- Report: [output-template.md](references/output-template.md)4344`Bash` is permitted only for read-only curl and observation queries reviewed against the safety rules. Missing endpoint allowlist means P3 is skipped.4546## P0 — Scope and Safety47481. Select `--env`, defaulting to test, and load its configuration.492. Run `scripts/health-probe.sh <health-url>` (3 attempts, 2-second timeout).503. Compare local HEAD with the deployed version and warn on mismatch.514. Confirm every planned operation is read-only.525. Determine L4/L3/L2-API/L2-OBS/L1 using the playbook matrix.5354API unreachable with logs gives L2-OBS; without logs gives L1. At L1 skip P3/P4. `--level L2` remains an alias for L2-API.5556## P1 — Diff-Lite Scope5758Follow [blackbox-testing.md § P1](references/blackbox-testing.md#p1-diff-lite-scoping). Map `git diff main...HEAD --name-only` to affected endpoints, dependency chains, active triggers, and passive targets. This phase scopes behavior; it does not judge code quality. If no diff exists, build scope from the user's feature description.5960## P2 — Test Charter and Approval6162Follow [blackbox-testing.md § P2](references/blackbox-testing.md#p2-test-charter-design). Generate only cases supported by the detected level: L1 regression, L2 active trigger, L3 passive observation, and M1 metrics. Present the charter and wait for user approval before any P3 request.6364## P3 — API Execution6566Run only for L2-API/L3/L4 after approval. For each allowlisted case:67681. Confirm method and endpoint exactly match the allowlist.692. Run one request at a time with fixed, non-PII test parameters:70 `scripts/api-exec.sh <GET|POST> <allowlisted-url> [json-payload]`713. Record its request ID, HTTP status, response fields, and latency.7273The script rejects methods other than GET and POST; the operator must separately confirm that POST is allowlisted and read-only. L2-OBS always skips P3.7475## P4 — Observation Correlation7677Follow [blackbox-testing.md § P4](references/blackbox-testing.md#p4-log-verification-flow).7879- L3/L4: correlate each request ID, then fall back to alternate fields and endpoint/time window.80- L2-OBS/L3/L4: scan the observation window for related errors and warnings.81- Observe background services after their expected delay; query affected metrics at L4.82- Record blind spots. A missing log signal is not itself proof of failure.8384For L2-OBS use deploy time → now, a user-provided window, or the last 30 minutes, in that order.8586## P5 — Independent Verdict87881. The primary model forms a conclusion from P3/P4 evidence.892. Only when the caller explicitly requests a second opinion, dispatch a fresh,90 isolated read-only reviewer with P1/P3/P4 evidence. The reviewer must assess91 scope, commands, allowlist compliance, evidence, blind spots, and confidence92 without receiving the primary conclusion. An explicit93 `--second-opinion=codex-exec` may supply an additional one-shot CLI review;94 it is additive and opt-in, with no hidden fallback.953. If no second opinion is requested or available, label the result96 **primary model only** and state plainly: "Only the primary97 model's verdict is present; no independent review ran." Do not describe the98 primary result as independently verified.994. Integrate available conclusions using the verdict/confidence rules in the100 playbook, keeping a skipped or unavailable reviewer visible.1015. Render [output-template.md](references/output-template.md).102103The report may recommend another skill, but must not auto-invoke it.104105## Output106107Produce the report from `references/output-template.md`. It must identify the deployed108target and local/deployed alignment, record the detected degradation level, list every109executed and skipped case with the reason, preserve request/log/metric evidence, state110blind spots, and end with the integrated verdict and confidence. A skipped API phase is111valid only when the level or safety gate requires it; it is not a passing execution.112113## Verification114115- [ ] P0 established environment, deployment alignment, read-only scope, and the116 degradation level.117- [ ] P1 mapped the diff or explicitly documented the feature-description fallback.118- [ ] P2 produced a level-appropriate charter and received approval before any P3119 request.120- [ ] Every P3 request matched the endpoint allowlist and recorded status, fields, and121 latency, or was marked skipped with a reason.122- [ ] P4 correlated logs or metrics when available and recorded the observation window123 and blind spots.124- [ ] P5 contains the primary analysis, any requested independent review (or the125 explicit primary-only state), and the integrated verdict from the126 playbook.127- [ ] The final report matches `output-template.md` and does not claim evidence for a128 phase that was skipped or unavailable.