KhurrumMahmood /find-test-obligation-drift
Use this before finishing a multi-file change. It inspects explicit
paths, staged files, a ref diff, or the working tree diff and reports
where the touched surface implies a verification tier that the diff does
not appear to cover. Product backend/UI obligations are inferred only
from the host's .engineering/docs/product-workflows.md descriptor. If
that descriptor is absent, the run still checks skill-script and tooling
obligations, but it must not claim product workflow coverage was
evaluated.
How success is judged
run.pywrites a report directory underreports/find-test-obligation-drift/scan-<UTC>/containingdetections.jsonl,findings.json, andreport.md, and updatesreports/find-test-obligation-drift/latest.- The console output is pasted in the handoff, including
workflow scope patterns: <N>,findings: <N>, andwrote <dir>. - If product backend/UI files are in scope, the run used at least one
descriptor pattern from
.engineering/docs/product-workflows.md; ifworkflow scope patterns: 0, the summary says product workflow obligations were not inferred. - Findings are treated as advisory obligations, not proof that tests failed. The user-facing verdict is "obligation present/missing" with the pasted detector artifact, never "verified" based on a claim. Write toward these gates from Stage 0.
Grade only by the report artifacts and real command output. A statement
that "the diff has enough tests" does not count unless report.md or
detections.jsonl backs it.
Pipeline
Stage 0 — Resolve target and descriptor context
Pick one input form:
- Explicit paths: pass them as positionals.
- Staged diff: pass
--staged. - Ref diff: pass
--changed-from REF. - No args: scans the working-tree diff.
Before interpreting product backend/UI absence, check the descriptor state. The script prints the descriptor-derived pattern count; zero means this shipped kit has no host product workflow scope to infer from.
Stage 1 — Run the detector
.venv/bin/python .claude/skills/find-test-obligation-drift/scripts/run.py <paths...>
.venv/bin/python .claude/skills/find-test-obligation-drift/scripts/run.py --staged
.venv/bin/python .claude/skills/find-test-obligation-drift/scripts/run.py --changed-from main
Supported run.py flags:
paths(zero or more): explicit files or directories.--project-root DIR: project root; defaults to current working directory.--staged: scangit diff --cached.--changed-from REF: scangit diff REF.--skip-effectiveness-log: write report artifacts without appending toreports/_meta/effectiveness.jsonl.
Reports are written under
reports/find-test-obligation-drift/scan-<UTC>/.
run.py returns exit 0 when report artifacts are written, even when it
finds obligations. Domain findings live in the report, not the process
exit code. detect.py and report.py are lower-level helpers; use
run.py for normal skill execution.
Stage 2 — Read the report and emit the declared verdict
Open the generated report.md and findings.json.
Dispatch by declared verdict:
- 0 findings, workflow scope patterns > 0: no drift found for the touched, descriptor-scoped surfaces. Still run the verification commands required by the actual task.
- 0 findings, workflow scope patterns = 0: no generic skill/tooling drift found; product backend/UI obligations were not inferred because the host descriptor is absent or empty.
- 1+ findings: list each detector band with file, recommendation, and obligation. These are advisory repair items; the owner either adds coverage or records a concrete reason the touched file did not need that tier.
Detector Bands
missing_backend_test_obligationmissing_ui_test_obligationmissing_skill_smoke_obligationmissing_quality_tool_test_obligation
Promote a band to a diff-scoped lint only after fixture coverage, at least one real fix, explicit false-positive handling, and low host workflow noise.
When things go sideways
| Symptom | Action |
|---|---|
git diff form fails because the ref is unknown |
Re-run with explicit paths or a valid --changed-from REF; paste the failing command output |
workflow scope patterns: 0 but the change is product backend/UI work |
State that the host descriptor is absent or empty; do not infer product obligations from baked-in paths |
| Report directory cannot be written | Stop and report the exact write failure; do not claim the detector ran |
detections.jsonl exists but report.md or findings.json is missing |
Treat the run as incomplete; re-run run.py once, then report the missing artifact if it persists |
| Findings look like docs/comment-only false positives | Keep the finding in the report and add the human reason in the handoff; do not delete or rewrite detector output |
| A product workflow descriptor has malformed rows | The descriptor parser skips malformed rows; if the pattern count is lower than expected, fix the descriptor before trusting absence of product findings |
Replay case
After material edits to this skill or its scripts, run the fixture smoke and paste the output:
.venv/bin/python .claude/skills/find-test-obligation-drift/scripts/smoke.py
Also prove the normal CLI remains executable:
.venv/bin/python .claude/skills/find-test-obligation-drift/scripts/run.py \
--project-root . \
--skip-effectiveness-log \
.claude/skills/find-test-obligation-drift/scripts/run.py