Regression Benchmark Orchestrator

Runs repeatable cross-workflow benchmarks to detect drift in grading stability, artifact quality, and policy compliance across model or prompt revisions.

alainlebret Updated

File contents

Regression Benchmark Orchestrator

Goal

Provide early warning when system changes degrade quality or consistency.

Inputs

  • Baseline benchmark cases
  • Candidate workflow outputs
  • Historical benchmark reports

Produce

Write:

  • regression-benchmark-report.json
  • regression-diff.md
  • optional benchmark-traces/

Benchmark dimensions

  • grade stability across repeated runs
  • quality gate pass-rate deltas
  • language-policy drift
  • artifact completeness drift
  • runtime and token-cost drift

Rules

  • Compare against explicit baseline version tags.
  • Flag statistically or operationally meaningful regressions.
  • Recommend rollback or investigation when required thresholds are breached.

alainlebret/claude-agents/tree/main/higher-ed-teaching-agents/skills/regression-benchmark-orchestrator commit 50637b4e7c

Frequently asked questions

npx skillmds@latest add alainlebret/regression-benchmark-orchestrator