zwright8 Governance Confidence Calibration Engine
Mission
We need this skill because high-speed autonomy needs enforceable guardrails to stay aligned. This specific skill aligns reported confidence with actual uncertainty.
Activation Cues
- Task requires calibration curves and error bins in Safety and Governance.
- Task needs explicit risk controls, approval gates, and traceable outcomes.
- Task output must include artifact handoff for humans and agents.
Execution Plan
- Define the scope and success metrics for
Governance Confidence Calibration Engine, including at least three measurable KPIs tied to unsafe actions and policy drift. - Design and version the input/output contract for policies, violations, and mitigation actions, then add schema validation and failure-mode handling.
- Implement the core capability using calibration curves and error bins, and produce calibrated confidence scores with deterministic scoring.
- Integrate the skill into swarm orchestration: task routing, approval gates, retry strategy, and rollback controls.
- Add unit, integration, and simulation tests that explicitly cover unsafe actions and policy drift, then run regression baselines.
- Deploy behind a feature flag, monitor telemetry/alerts for two release cycles, and iterate thresholds based on observed outcomes.
Runbook
Preflight:
- None specified.
Execution:
- None specified.
Recovery:
- None specified.
Handoff:
- None specified.
Guardrails
- [quality] Require validations before promoting outputs.
Success Metrics
- Primary metric: unsafe actions
- Secondary metrics: policy drift, decision drift
- Review cadence: weekly
Output Contract
- Return a concise execution summary with key decisions.
- Return risk and mitigation notes with unresolved blockers.
- Return artifact target:
calibrated confidence scores. - Return recommended follow-up tasks for next wave execution.