Regression Benchmark Orchestrator
Goal
Provide early warning when system changes degrade quality or consistency.
Inputs
- Baseline benchmark cases
- Candidate workflow outputs
- Historical benchmark reports
Produce
Write:
regression-benchmark-report.jsonregression-diff.md- optional
benchmark-traces/
Benchmark dimensions
- grade stability across repeated runs
- quality gate pass-rate deltas
- language-policy drift
- artifact completeness drift
- runtime and token-cost drift
Rules
- Compare against explicit baseline version tags.
- Flag statistically or operationally meaningful regressions.
- Recommend rollback or investigation when required thresholds are breached.