Mine Ledger

Mine Ledger

Paths. analysis-results/… and progress-tracker/… in this skill are the default workspace layout. They resolve through locations.yaml in $TRAUST_CONFIG_HOME (docs/setup.md, Storage locations); substitute your configured roots.

Turn the campaign's disposition ledger into rule-authoring fuel: extract every ledger-confirmed true positive from the cumulative *-findings-current.json reports, cluster by (primary CWE, language), match against the harness-authored opengrep pack, and aggregate the per-fact judge decisions audits record in scanner_correlation into campaign-wide per-rule precision.

This skill is the out-of-band half of a loop the audits feed by default: /secure-code-audit's pre-scan records every promote/dismiss decision as a structured scanner_correlation entry (tool opengrep, rule_id, result: promoted|dismissed), and this skill harvests them.

Input

$ARGUMENTS (all optional):

• A findings tree to mine — default analysis-results/findings resolved from the campaign workspace (or AUDIT_RESULTS_ROOT).
• --pack <dir> — rule pack to check coverage against (default: the in-repo pack harnessing/3-audit/secure-code-audit/opengrep-rules/).
• --out <dir> — output directory. Default for campaign runs: progress-tracker/metrics/rule-mining/ so the report versions alongside the other dashboards.

Bundled precedent cards

precedent-cards/pilot.yaml supplies the pilot cards used by tests/test_compile_precedent_cards.py to check card validation and compiler behavior. Consult it when maintaining precedent-card compilation; these fixtures are not findings or production sweep inputs.

Procedure

Step 1 — run the deterministic miner (all analysis is script-side):

python3 -m traust.cli sweep mine \
  --root analysis-results/findings \
  --out progress-tracker/metrics/rule-mining

Outputs: tp-corpus.jsonl (one confirmed TP per line — the rule specification material), rule-mining.json, rule-mining.md.

Optional accelerator — findings-db pre-queries. Cluster-shape questions ("which CWE×language pairs dominate confirmed TPs?", "how many resolved-at-fix-commit findings exist for CWE-78?") can be answered instantly from the C9 projection (/findings-db) before committing to a full mine. The miner itself stays file-based — its output is rule-authoring material and must reflect the live ledgers, not a projection that may be one census behind. DB for scoping, miner for the corpus of record.

Step 2 — interpret the backlog. For the top uncovered clusters, read 3–5 of the example findings (ids are in the report; fetch their entries from the source reports) and answer, per cluster: is this mechanically detectable? Clusters like missing-authentication (CWE-306) or confused deputy (CWE-441) usually are not — say so and move on. For detectable clusters, propose concrete rule candidates: language, sources/sinks or pattern shape, expected sanitizers, and the example finding ids that specify the behavior. That proposal — not rule YAML — is this skill's deliverable; authoring follows the pack README's standards and its test-and-calibrate gate.

Step 3 — review precision. Precision only counts reports where the pre-scan actually ran: check metadata.additional.deterministic_steps ("opengrep": "ran" vs "skipped: …") before treating an absence of judge decisions as signal — a skipped scanner is not a quiet rule. Any rule whose campaign precision (from audits' judge decisions) sits below the plan's ~50% gate is flagged: read a sample of its dismissal rationales from the source reports and propose a tightening (sanitizer, metavariable constraint, benign-suffix exclusion) or retirement.

Step 4 — hand back. Report: TP corpus size, top uncovered clusters with a detectability verdict each, the per-language coverage table (confirmed TPs, covered/uncovered clusters, pack rule count — sorted by uncovered-TP count, the tranche-priority order), rule candidates proposed, precision flags, and the calibration-worklist size (repos whose confirmed TPs the pack should rediscover — the re-scan set for validating pack changes). Update the rule-pack backlog your deployment keeps when the picture has changed materially.

Regression-rule drafts (C6)

Every RESOLVED-at-a-fix-commit ledger finding is a free calibration pair: pre-fix file (rule must fire) and post-fix file (rule must not). Stage drafts, author, verify, promote:

# 1. stage drafts (deterministic; identical pairs skipped)
python3 harnessing/mine-ledger/scripts/emit_rule_drafts.py --results-root ../analysis-results \
    [--cwe CWE-494] [--limit 6]
# 2. author patterns: in the draft's rule.skeleton.yaml so that
python3 harnessing/mine-ledger/scripts/emit_rule_drafts.py \
    --verify harnessing/mine-ledger/rule-drafts/<draft> \
    --rule   harnessing/mine-ledger/rule-drafts/<draft>/rule.skeleton.yaml
# passes (before/ fires, after/ silent)
# 3. promote: move the rule into the pack's matching CATEGORY file
#    (never a regression.yaml dump) with ruleid:/ok: fixture lines,
#    keep the regression_of metadata (fingerprint + fix commit — that is
#    what links a future firing back to the original resolved finding),
#    then delete the draft dir.

Not every fix is rule-expressible: additive-coverage fixes (a filter gained patterns) and cross-file behavioral fixes have no matchable pre-shape — reject those drafts with a note rather than authoring a noisy rule. First promoted regression rule: traust-bash-regression-codecov-uploader-latest (bash/supply-chain).

Class-generalization sweeps (plan §6, Phase 4)

Every confirmed finding with a syntactic signature — a sink call form, a config key+value, an annotation misuse, a hardcoded literal — defines a class the whole corpus should be checked for. The sweep engine (python3 -m traust.cli sweep) extends the backlog flow above into a full loop: backlog item → draft → calibrate → SWEEP → triage. Four resumable stages:

# 1. gather + classify confirmed findings (ledger validity=confirmed
#    plus the Phase-0 confirmation artifacts: fast-track criticals,
#    matrix stage-2 votes, 0d fresh-only triage); rule-expressible vs
#    not is decided deterministically and every "not" carries a reason
python3 -m traust.cli sweep collect
# 2. stage candidate rules for expressible classes with no harness-pack
#    coverage (dedup against pack rule ids + existing drafts); drafts
#    land in rule-drafts/ per the conventions above, each citing its
#    source confirmed finding(s) in metadata.generalized_from
python3 -m traust.cli sweep draft
# 3. author the pattern (human+LLM, same authoring contract as C6),
#    calibrate via the pack's test-and-calibrate gate, then sweep —
#    shallow clones into a bounded temp root, one rule, --limit capped
python3 -m traust.cli sweep sweep --rule <draft-or-rule-id>
# 4. aggregate hits into the triage-ready candidates file + summary
python3 -m traust.cli sweep emit --rule <draft-or-rule-id>

Run-by-default policy (plan §6): once a class rule is authored and calibrated, the corpus-wide sweep RUNS by default — it is not a proposal to maybe sweep later — and its results ENTER TRIAGE. The artifact says so in its own header. The engine never files findings, never routes to Jira, and never adjudicates a hit.

Human points, unchanged: (1) pattern authoring against the cited source findings; (2) rule promotion into the shipped pack, which stays with the existing mine-ledger calibration path (fixture ruleid:/ok: lines, the plan's test-and-calibrate gate — never promoted by the engine); (3) verdicts on sweep hits, which belong to /triage and its disposition ledger like any other candidate batch.

Sweep output lands at analysis-results/scan-testing/sweeps/<rule-id>/sweep-candidates.json (schema: repo, file:line, matched excerpt, rule id, source-finding provenance) plus a sweep-summary.md, and is consumed as /triage input material. Stage state lives under the sweeps root's _state/ dir; every stage skips work already done (re-run to resume).

Scheduled lane (python3 -m traust.cli sweep rule-lane)

The deterministic half of this skill runs as a standing lane (docs/continuous-operations.md, "Lane cadences") — orchestrator-neutral in the dependency-watch pattern: identical behavior from an operator session, cron, or any enterprise scheduler. One entry point chains the miner, sweep_engine.py collect + draft, bounded regression-draft staging (emit_rule_drafts.py --skip-existing --limit N — existing draft dirs are never clobbered), and a delta report against the previous rule-mining.json (new/resolved/changed uncovered clusters, precision movements crossing the ~50% gate, TP-corpus growth) written to progress-tracker/metrics/rule-mining/lane-delta.{json,md}. Exit 0 = quiet, 1 = attention needed (the delta says what), 2 = a stage failed. No model calls happen inside the runner.

Stage-A gate reachability. Each pass also answers, per watched language and candidate pack, is calibrating this language possible yet? — no clones, pure corpus plus the cached pack. A rule clears the rediscovery gate only by firing in ≥3 repos that share a CWE the pack targets, so when no covered CWE spans that many the run is unwinnable before it starts and would return a table of structural zeros. Measured 2026-08-07 against argus-observe-rules: rust/c/csharp max span 1, cpp 0 — all blocked on ground truth, not tooling. The lane raises attention only when a gate opens; a steady blocked state is silent, because a weekly complaint about a known-blocked language is noise. Watched set: WATCHED_LANGS / WATCHED_PACKS in the runner.

Enabled external-pack rules are re-checked against the precision gate. An external pack is enabled per-rule via $TRAUST_CONFIG_HOME/rule-pack-allowlist.yaml (run_opengrep.py --rule-allow @<file>), never wholesale — argus ships 640 rules of which 19 are 86% of volume. Stage A can only measure rediscovery before a rule has ever run; precision arrives afterwards from scanner_correlation, so each pass re-scores every allowlisted rule and flags any that fell below the ~50% gate as a demotion candidate (one YAML line). Rules not yet judged in any audit are listed separately — silence is not a pass.

# weekly (Mondays 06:40), from the campaign workspace root — plus
# on-demand whenever /drift-watch's `rule-mining` row goes stale
# (ledger events >7d newer than the artifact, the post-wave trigger)
40 6 * * 1  cd $WS && python3 -m traust.cli sweep rule-lane

Standing delegation (user, 2026-07-29): scheduled agent runs of this skill proceed from the lane's delta report through rule authoring and the full test-and-calibrate gate — fixture ruleid:/ok: lines firing/silent, the whole-pack regression suite, TP-rediscovery calibration against the corpus repos — WITHOUT asking for per-run approval. The mechanical gate is the promotion criterion: a rule that passes all three legs is promoted into the pack; one that does not is not, and no amount of session judgment overrides that in either direction. Sweep hits and mined candidates still route to /triage like any other candidate batch — the lane never auto-files findings, never writes to ledgers, and never touches Jira.

Constraints

• Read-only over the findings tree; never edits reports or ledgers.
• The miner is a candidate generator: cluster counts and precision numbers route attention, they do not conclude — rule decisions stay with the authoring loop and its mechanical calibration gate (delegated for scheduled runs as above).
• Cadence: weekly via the lane runner (python3 -m traust.cli sweep rule-lane, see Scheduled lane), plus a post-wave trigger — /drift-watch's rule-mining row goes stale when ledger changes are >7 days newer than rule-mining.json, so a large audit/triage wave summons the lane ahead of schedule.

Integrations

• Inputs: cumulative *-findings-current.json ledgers and the scanner_correlation judge decisions audits record by default.
• Inputs (sweeps): any prior confirmation artifacts under analysis-results/scan-testing/ (fast-track critical confirmations, matrix votes, fresh-only triage) alongside the ledgers.
• Outputs: rule-mining.{json,md} inform the opengrep-pack backlog (plan doc) and are freshness-gated by /drift-watch's rule-mining row; tp-corpus.jsonl is the rule-authoring specification corpus consumed by the pack's test-and-calibrate workflow — a human authoring loop, deliberately not another skill; sweep runs write sweep-candidates.json and sweep-summary.md per rule under analysis-results/scan-testing/sweeps/, consumed by /triage as raw candidate input (run-by-default policy, error-correction plan §6; the ingest mapping — candidates[] fields, origin: sweep, provenance-as-context — is the triage SKILL.md's "Note — class-generalization sweep candidates").
• Outputs (lane): lane-delta.{json,md} (producer: python3 -m traust.cli sweep rule-lane) — consumed by the operator / scheduled /mine-ledger session as the worklist for the delegated authoring-and-calibration pass (Scheduled lane above), and by /drift-watch's rule-mining staleness row, which triggers the next lane run when ledgers outrun the artifact. The lane's nonzero exit is the scheduler-facing attention signal.