yogsoth-ai Leaderboard Dynamics Analysis SOP
Analyze the dynamics of benchmark leaderboards: score distributions, compression over time, selective reporting patterns, and statistical artifacts.
Input
- leaderboard_data: Structured data with (model_name, score, date, optional: model_size, paper_reference)
Procedure
- Compute score distribution statistics over time
- Detect score compression (shrinking gap between top models)
- Identify selective reporting patterns (cherry-picking, subset selection)
- Analyze relationship between model size and score
- Detect anomalies (withdrawn results, unreplicated claims)
Output
Leaderboard health assessment with statistical evidence.
Available SOPs
Optional, no fixed order; the final leaf is always a sop.
| SOP | When to use |
|---|---|
| spawn-agent | Spawn a customized CC subagent with full MCP tool access. Used by SOPs that declare execution: subagent. |