Regression Benchmark Orchestrator

Runs repeatable cross-workflow benchmarks to detect drift in grading stability, artifact quality, and policy compliance across model or prompt revisions.

alainlebret Updated 0 repo stars

File contents

Regression Benchmark Orchestrator

Goal

Provide early warning when system changes degrade quality or consistency.

Inputs

  • Baseline benchmark cases
  • Candidate workflow outputs
  • Historical benchmark reports

Produce

Write:

  • regression-benchmark-report.json
  • regression-diff.md
  • optional benchmark-traces/

Benchmark dimensions

  • grade stability across repeated runs
  • quality gate pass-rate deltas
  • language-policy drift
  • artifact completeness drift
  • runtime and token-cost drift

Rules

  • Compare against explicit baseline version tags.
  • Flag statistically or operationally meaningful regressions.
  • Recommend rollback or investigation when required thresholds are breached.

alainlebret/claude-agents commit 50637b4e7c

Frequently asked questions

npx get-skill add alainlebret/regression-benchmark-orchestrator