Conformance Loop

Iteratively improve Fallow analysis accuracy by comparing it with competing tools and verified source truth across a stable real-world corpus.

fallow-rs 47e2239 900 B Updated

File contents

Conformance loop

  1. Define the capability and stable project corpus.
  2. Run Fallow and comparison tools with documented equivalent settings.
  3. Manually verify disagreements against source.
  4. Classify true positive, false positive, false negative, or model difference.
  5. Record the classification in tests/conformance/adjudications.json before writing the fix. A verdict that lives only in a session or in .plans/ is lost, and the daily lane then re-reports the same disagreement as unadjudicated forever.
  6. Implement one general correction with a regression fixture.
  7. Re-run the full corpus and retain only net improvements.
  8. Run review.

Competitor output is a lead, not ground truth.

fallow-rs/fallow/tree/main/.agents/skills/conformance-loop commit 47e2239731

Frequently asked questions

npx skillmds@latest add fallow-rs/conformance-loop