Leaderboard Dynamics Analysis

Analyze leaderboard score distributions, compression, selective reporting

yogsoth-ai Updated 512 repo stars
No DARE runtime: 271 SKILL.md files executed by any capable agent, instead of a seven-layer framework stack

File contents

Leaderboard Dynamics Analysis SOP

Analyze the dynamics of benchmark leaderboards: score distributions, compression over time, selective reporting patterns, and statistical artifacts.

Input

  • leaderboard_data: Structured data with (model_name, score, date, optional: model_size, paper_reference)

Procedure

  1. Compute score distribution statistics over time
  2. Detect score compression (shrinking gap between top models)
  3. Identify selective reporting patterns (cherry-picking, subset selection)
  4. Analyze relationship between model size and score
  5. Detect anomalies (withdrawn results, unreplicated claims)

Output

Leaderboard health assessment with statistical evidence.

Available SOPs

Optional, no fixed order; the final leaf is always a sop.

SOP When to use
spawn-agent Spawn a customized CC subagent with full MCP tool access. Used by SOPs that declare execution: subagent.

yogsoth-ai/de-anthropocentric-research-engine commit a7e0953f66

Frequently asked questions

npx get-skill add yogsoth-ai/leaderboard-dynamics-analysis