Workflow Verify Before Filing

Verify Before Filing

Operational scaffolding ships beside this file, not inside it: Phases 1–2 as workflows/verify-before-filing.workflow.js (agent prompts, schemas, gate logic), Phase 3 as the executable scripts/file-wave.sh. The rationale for both — why each constant is what it is — plus the worked example is in REFERENCE.md. This file is the decision layer.

A backlog of upstream bug candidates — audit docs, "file this later" notes, workaround commits — is a list of hypotheses dated to when they were observed, not a filing queue. Upstream moved since: versions shipped, files restructured, other deployers reported the same thing, and some of your own diagnoses were wrong. Filing the backlog as-is produces duplicate and already-fixed reports — exactly the noise that makes maintainers stop reading your issues. Verify every claim at upstream HEAD, dedup against the trackers (including your own earlier reports), and only file what survives.

Measured base rate (FVH → SIMPL-Open, 2026-06-11): of 24 accumulated candidates, only 12 were real-and-current — 7 claims were invalid on inspection, 3 were already fixed upstream, 1 was obsolete, 1 duplicated our own earlier report's by-catch. Half the backlog would have been noise.

When to Use This Skill

The Pipeline

Phases 1–2 also ship as a bundled harness — see Workflow harness (template) below.

Phase 0 — Consolidate a candidate manifest

One JSON/table entry per candidate: id, the claim (precise, falsifiable), target upstream project, version observed, source refs (your commits/PRs that hold real error output), and known-filed prior reports to dedup against. Merge all sources first — audit docs, strategy docs, and git sweeps usually overlap. Shape:

{
  "id": "W2-13",
  "slug": "notification-smtp-ec-defaults",
  "claim": "Chart defaults SMTP to dev@simpl-europe.eu via ssl0.ovh.net (vendor dev infra) as live default values; should be placeholder/required.",
  "targets": ["group/subgroup/notification-service"],
  "observed_version": "2.1.1 (Apr 2026)",
  "sources": ["audit-doc item 6"],
  "evidence_prs": [1826]
}

Keep prior-filed report URLs (with issue iids) in the same manifest so search agents can fetch their bodies.

Phase 1 — Verify + dedup (two agents per candidate, parallel)

Verify agent (read-only against upstream): fetch the implicated files at default-branch HEAD and the latest tag; quote the current content; return a verdict from a closed vocabulary:

still-present | partially-fixed | fixed-upstream | obsolete-version
| claim-invalid | could-not-verify

plus targetProject, quoted evidence, checkedRefs, and notes (files moved, versions drifted, framing corrections). Hard rule: agents are read-only upstream — GET requests only; nothing writes until the filing phase. State that rule verbatim in every agent prompt.

Both prompts, with the forge-tooling block and the schemas, are the VERIFY_PROMPT / SEARCH_PROMPT constants in workflows/verify-before-filing.workflow.js — adapt those rather than retyping them.

Gate precedence: any duplicate kills the filing regardless of verdict; could-not-verify never files (record a human follow-up task instead).

Search agent: tracker search (issues + MRs, all states, several phrasings including exact error strings) on the target project and group-wide — plus fetch the full bodies of your own prior reports and check overlap including their by-catch findings. Self-duplicates are the embarrassing kind.

Gate: only still-present/partially-fixed with no duplicate proceeds. Everything else gets a recorded disposition — that record is a deliverable, not waste (see Phase 4).

Phase 2 — Draft to a house template

Per surviving candidate, one markdown file per issue:

# <symptom-first title — becomes the issue title>
<!-- target: <project path>  (stripped by the filing script) -->

## Summary
<claim, with evidence as blob links PINNED to the verified refs
(https://<forge>/<path>/-/blob/<ref>/<file>#L<n>) — not bare paths,
not `main` if HEAD drifts>

<real error signature mined from your own incident PRs/logs>

## Suggested fix
<EXACTLY ONE recommended fix; alternatives get one trailing sentence;
"happy to open the MR" only when trivial>

---
Observed while <one-line deployment context>; verified against <refs> on <date>.

Never leak internal PR numbers or repo paths into the body — use them only to mine evidence. Then gate every draft through agent-patterns-plugin:cold-read-gate (isolated haiku maintainer cold-read; one revise round, re-gate only if the verdict was needs-revision).

Phase 3 — Paced filing

Issue-creation endpoints rate-limit aggressively (observed: a GitLab instance returning 429 after a single create). Never loop gh/glab issue create by hand — write the Phase 1+2 result array to a JSON file and run the script, which owns the pacing, the retries, and the URL manifest:

bash "${CLAUDE_SKILL_DIR}/scripts/file-wave.sh" --results /abs/path/results.json --dry-run
bash "${CLAUDE_SKILL_DIR}/scripts/file-wave.sh" --results /abs/path/results.json --forge glab --host <instance>

Run the real invocation with Bash run_in_background: true — at 70 s per create a wave outlives a foreground call. Dry-run first: it resolves every title and target project and creates nothing.

The script files only disposition: "file" entries, exits 0 on an empty set, appends every URL and every failure to filed-urls.txt, and continues past a failure rather than aborting the batch. Forge dispatch is deterministic: --forge gh|glab > $FILE_WAVE_FORGE > a set GITLAB_HOST (implies glab) > gh. --help documents every flag; the pacing rationale and the invariants its tests pin are in REFERENCE.md. Cross-link related new issues afterwards (also paced). Created GitLab issues may surface as /-/work_items/ URLs.

Phase 4 — Bookkeeping (the dispositions are deliverables)

• Annotate the source docs the candidates came from: filed URL, fixed-upstream (version), duplicate-of, obsolete, or claim-retracted — the audit trail keeps stale claims from being re-filed next quarter.
• Fixed-upstream discoveries usually imply local action: a fork you can retire, a pin you can advance, a workaround you can delete. Record each as a follow-up task.
• Post the disposition table to your tracking issue; close it if nothing known remains unfiled.

Workflow harness (template)

workflows/verify-before-filing.workflow.js ships beside this skill. It is a TEMPLATE to adapt, not a script to run verbatim. Read it, then rewrite it for the work in front of you. It covers Phases 1–2 only; Phase 3 is scripts/file-wave.sh, whose input contract is the harness's return value.

Adapt freely: the agent prompts and their forge-tooling block (the shipped one is GitLab), the wave width, the house draft template, the effort tiers, and the search phrasings.

Preserve across any adaptation: (a) the loop bound comes from the candidate manifest passed in as args, never from a prose "for each" — including the ≤5 read wave, which paces reads the way file-wave.sh paces writes; (b) the closed verdict vocabulary and the gate ['still-present','partially-fixed'].includes(verdict) && duplicateFound === 'no', in that precedence — a duplicate kills the filing regardless of verdict, and could-not-verify never files; (c) two barriers — the intra-candidate parallel([verify, search]), because the gate reads both, and the batch-dedup pass, which compares survivors to each other rather than only to the tracker. Also structural: the cold-read agent is never the drafter (that independence is the gate), exactly one revise round, and DRAFT_SCHEMA carries the issue body — a workflow script has no filesystem, so dedup cannot merge on a path.

Skip the harness when: the manifest holds one or two candidates — that is a linear pass and the harness is pure overhead (the template aborts below three). A 24-candidate run is roughly 100–140 agents. The steps above remain the authoritative description of what each stage must produce; the harness only fixes how the work is split.

Two clauses this template carries. The second is unconditional here — this skill's entire output is a forge mutation:

Never Workflow({resumeFromRunId}) to retry a few failed worktree agents — a resume re-runs agents that already succeeded and opens duplicate PRs (#1868). Re-dispatch the failed units fresh and sequentially after checking gh pr list --head <branch> --state all --json number,state.

Push, PR creation, and GitHub mutations happen only in the single sequential finalise stage, never inside a fanned-out agent. Here that stage is Phase 3 (scripts/file-wave.sh): every agent in the harness is read-only upstream, and the harness returns data for the script to file.

Verdict Vocabulary Notes

claim-invalid is not failure — it's the workflow catching your own docs drifting from reality. Correct the doc in the same pass.

Common Mistakes

Related

• agent-patterns-plugin:cold-read-gate — the pre-publish legibility gate (Phase 2)
• agent-patterns-plugin:verify-before-plan — same epistemics one level up: premises decay; check before acting on them
• workflow-preflight — remote-state verification before implementation work, the in-repo sibling
• User rule verify-upstream-before-patching (where present) — the single-item inline form of Phase 1