# Saturation Analysis

> Track score trajectories, detect saturation/failure points — 15 benchmarks, 50 papers, 60 web searches

- Skill: `yogsoth-ai/saturation-analysis` (Agent Skill)
- Install (CLI): `npx skillmds@latest add yogsoth-ai/saturation-analysis`
- Raw SKILL.md: https://api.skillmd.com/api/skills/yogsoth-ai/saturation-analysis/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: yogsoth-ai (https://skillmd.com/u/yogsoth-ai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/yogsoth-ai/saturation-analysis

---


# Saturation Analysis Strategy

Track benchmark score trajectories over time to detect saturation signals, ceiling effects, and inflection points that indicate a benchmark has lost discriminative power.

## Purpose

Determine which benchmarks are approaching or have reached saturation, quantify remaining headroom, estimate time-to-ceiling, and identify the specific failure modes that remain unsolved even at high aggregate scores.

## Budget

| Resource | Floor | Target |
|----------|-------|--------|
| Benchmarks analyzed | 10 | 15 |
| Papers read | 35 | 50 |
| Web searches | 40 | 60 |

## State Ledger

```
<HARD-GATE>
| Metric | Current | Target | Status |
|--------|---------|--------|--------|
| Benchmarks analyzed | 0 | 15 | PENDING |
| Score trajectories built | 0 | 15 | PENDING |
| Papers fetched | 0 | 50 | PENDING |
| Papers read | 0 | 35 | PENDING |
| Web searches | 0 | 60 | PENDING |
| Saturation detections run | 0 | 15 | PENDING |
| Leaderboard analyses done | 0 | 10 | PENDING |
| Failure mode catalogs built | 0 | 5 | PENDING |
</HARD-GATE>
```

Cannot exit until 80% of all targets met.

## Available Tactics

- **score-trajectory-analysis** — Collect historical scores, fit saturation curves, detect inflection points

## Available SOPs

- **benchmark-inventory** — Identify target benchmarks in domain
- **metric-decomposition** — Analyze metric properties (ceiling effects, granularity)
- **leaderboard-dynamics-analysis** — Analyze score distributions and compression
- **saturation-detection** (shared from literature-survey) — Detect saturation signals
- **benchmark-synthesis** — Produce cross-benchmark saturation report

## Execution Guidance

1. **Inventory Phase**: Use benchmark-inventory to identify 15 benchmarks spanning different maturity levels
2. **Data Collection** (per benchmark):
   a. Search leaderboards (Papers With Code, official sites) for historical scores
   b. Collect papers reporting SOTA results chronologically
   c. Note human baselines, random baselines, and theoretical ceilings
3. **Trajectory Analysis** (per benchmark):
   a. Run score-trajectory-analysis tactic to build time-series and fit curves
   b. Run saturation-detection to classify saturation status
   c. Run leaderboard-dynamics-analysis for score compression analysis
4. **Failure Mode Mining**: For saturated benchmarks, identify remaining hard subsets
5. **Synthesis**: Cross-benchmark comparison of saturation timelines and patterns

## Output Format

```yaml
saturation_report:
  benchmark_name: string
  saturation_status: pre-saturation|approaching|saturated|supersaturated
  current_sota: float
  human_baseline: float
  theoretical_ceiling: float
  headroom_remaining: float
  estimated_time_to_ceiling: string  # e.g., "6-12 months"
  inflection_points: list[{date, score, cause}]
  score_compression: float  # top-10 score range
  remaining_hard_subsets: list[string]
  successor_benchmarks: list[string]
```

<!-- BEGIN available-tables (generated) -->

## Available Tactics

Optional, no fixed order; the final leaf is always a sop.

| Tactic | When to use |
| --- | --- |
| score-trajectory-analysis | Collect historical scores, fit saturation curves, detect inflection points |

## Available SOPs

Optional, no fixed order; the final leaf is always a sop.

| SOP | When to use |
| --- | --- |
| benchmark-synthesis | Produce final structured audit report |
| knowledge-acquisition-benchmark-inventory | Identify and catalog all relevant benchmarks in target domain |
| knowledge-acquisition-saturation-detection | Determine when additional searching yields diminishing returns. Analyzes the latest expansion batch against existing corpus to judge continue/near-saturation/saturated. Used by snowball and systematic-survey. |
| leaderboard-dynamics-analysis | Analyze leaderboard score distributions, compression, selective reporting |
| metric-decomposition | Decompose composite metrics into constituent signals, analyze polarity and ceiling effects |

<!-- END available-tables (generated) -->