# Threshold Calibration

> Systematically sweep consensus thresholds to observe which items achieve consensus at what level, producing a threshold-consensus curve.

- Skill: `yogsoth-ai/threshold-calibration` (Agent Skill)
- Install (CLI): `npx skillmds@latest add yogsoth-ai/threshold-calibration`
- Raw SKILL.md: https://api.skillmd.com/api/skills/yogsoth-ai/threshold-calibration/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: yogsoth-ai (https://skillmd.com/u/yogsoth-ai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/yogsoth-ai/threshold-calibration

---


# Threshold Calibration

Systematically vary the consensus threshold to understand the sensitivity of consensus classification. Rather than picking a single arbitrary threshold, sweep across a range to see which items are robust consensus (agree at any threshold) vs. fragile (only consensus at lenient thresholds).

## Stages

1. **Sweep** — Run `threshold-sweep` to compute consensus status at multiple threshold levels
2. **Classify** — Run `consensus-classification` to categorize items at the chosen operating threshold
3. **Measure** — Run `consensus-measurement` to validate final consensus scores

## Available SOPs

| SOP | Role in Tactic |
|-----|---------------|
| threshold-sweep | Compute consensus at multiple threshold levels, produce curve |
| consensus-classification | Classify items as consensus/dissensus at operating threshold |
| consensus-measurement | Validate final consensus scores with appropriate method |

## Execution Guidance

- Sweep range should cover 50%–90% agreement (or IQR 0.5–2.0)
- Identify "knee" in the curve where many items flip classification
- Robust consensus items (agree at strict thresholds) are highest confidence
- Fragile items (only consensus at lenient thresholds) need flagging
- Report both the curve and the classification at the chosen operating point

## Minimum Yield

- Threshold-consensus curve (threshold vs. number-of-consensus-items curve)
- Classification results (classification at operating threshold: consensus items, dissensus items)

<!-- BEGIN available-tables (generated) -->

## Available SOPs

Optional, no fixed order; the final leaf is always a sop.

| SOP | When to use |
| --- | --- |
| consensus-classification | Classify items as consensus or dissensus at a given threshold. |
| consensus-measurement | Compute consensus score from collected judgments using the appropriate statistical method. |
| threshold-sweep | Compute consensus status at multiple threshold levels to produce a threshold-consensus curve. |

<!-- END available-tables (generated) -->