# Performance Extraction

> Systematically extract performance data and conditions from papers — 30 methods, 150 data points, 40 web searches budget

- Skill: `yogsoth-ai/performance-extraction` (Agent Skill)
- Install (CLI): `npx skillmds@latest add yogsoth-ai/performance-extraction`
- Raw SKILL.md: https://api.skillmd.com/api/skills/yogsoth-ai/performance-extraction/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Finance & Business, Coding & Dev Tools
- Author: yogsoth-ai (https://skillmd.com/u/yogsoth-ai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/yogsoth-ai/performance-extraction

---


# Performance Extraction


## Purpose

Extract structured performance data from papers, leaderboards, and reproducibility studies. Each data point is a (Task, Dataset, Metric, Score, Conditions) tuple with full provenance. Prioritizes primary sources (original papers) but cross-references against leaderboards and third-party reproductions.

## Budget

| Resource | Floor | Target |
|----------|-------|--------|
| Methods covered | 20 | 30 |
| Data points extracted | 100 | 150 |
| Web searches | 25 | 40 |
| Papers read | 15 | 30 |

## State Ledger

```
<HARD-GATE>
| Metric | Current | Target | Status |
|--------|---------|--------|--------|
| Methods covered | 0 | 30 | BLOCKED |
| Data points extracted | 0 | 150 | BLOCKED |
| Web searches used | 0 | 40 | — |
| Papers read | 0 | 30 | — |
| Datasets covered | 0 | 5 | — |
| Metrics tracked | 0 | 3 | — |
</HARD-GATE>
```

Cannot exit until data_points >= 120 (80% of target).

## Available Tactics

- **leaderboard-harvesting** — Bulk data collection from structured platforms

## Available SOPs

- **score-extraction** — Extract tuples from individual papers
- **condition-cataloging** — Record conditions alongside scores

## Execution Guidance

1. For each method in the inventory, locate the original paper
2. Use score-extraction SOP on each paper to pull all reported results
3. Cross-reference against Papers With Code / benchmark leaderboards
4. Use condition-cataloging to record experimental setup for each score
5. Flag scores that lack essential condition information
6. Track provenance: which table/figure in which paper
7. Prefer results from official implementations over third-party

## Output Format

```json
{
  "data_points": [
    {
      "method": "string",
      "task": "string",
      "dataset": "string",
      "split": "test|val|dev",
      "metric": "string",
      "score": 0.0,
      "confidence_interval": [0.0, 0.0],
      "conditions": {
        "hardware": "string",
        "training_data_size": "string",
        "hyperparams_reported": true,
        "seeds_reported": true,
        "compute_budget": "string"
      },
      "provenance": {
        "paper_id": "string",
        "table_or_figure": "string",
        "is_primary_source": true
      }
    }
  ],
  "coverage_summary": {
    "methods_covered": 0,
    "datasets_covered": 0,
    "metrics_tracked": [],
    "missing_data_flags": []
  }
}
```

<!-- BEGIN available-tables (generated) -->

## Available Tactics

Optional, no fixed order; the final leaf is always a sop.

| Tactic | When to use |
| --- | --- |
| leaderboard-harvesting | Systematically collect performance data from platforms and papers |

## Available SOPs

Optional, no fixed order; the final leaf is always a sop.

| SOP | When to use |
| --- | --- |
| condition-cataloging | Record evaluation conditions (data splits, hyperparams, hardware, seeds) from a paper |
| score-extraction | Extract (Task, Dataset, Metric, Score, Conditions) tuples from a paper |

<!-- END available-tables (generated) -->