zwright8 Tooling Regression Sentinel
Mission
We need this skill because automation collapses when tools are flaky and failure modes are opaque. This specific skill prevents unnoticed quality drift after updates.
Activation Cues
- Task requires baseline-delta detection in Tool Reliability and Execution Quality.
- Task needs explicit risk controls, approval gates, and traceable outcomes.
- Task output must include artifact handoff for humans and agents.
Execution Plan
- Define the scope and success metrics for
Tooling Regression Sentinel, including at least three measurable KPIs tied to silent failures and cascading retries. - Design and version the input/output contract for tool runs, error signatures, and retry outcomes, then add schema validation and failure-mode handling.
- Implement the core capability using baseline-delta detection, and produce regression watchlists with deterministic scoring.
- Integrate the skill into swarm orchestration: task routing, approval gates, retry strategy, and rollback controls.
- Add unit, integration, and simulation tests that explicitly cover silent failures and cascading retries, then run regression baselines.
- Deploy behind a feature flag, monitor telemetry/alerts for two release cycles, and iterate thresholds based on observed outcomes.
Runbook
Preflight:
- None specified.
Execution:
- None specified.
Recovery:
- None specified.
Handoff:
- None specified.
Guardrails
- [quality] Require validations before promoting outputs.
Success Metrics
- Primary metric: silent failures
- Secondary metrics: cascading retries, decision drift
- Review cadence: weekly
Output Contract
- Return a concise execution summary with key decisions.
- Return risk and mitigation notes with unresolved blockers.
- Return artifact target:
regression watchlists. - Return recommended follow-up tasks for next wave execution.