# Math Reasoning Eval

> Evaluates the mathematical reasoning capabilities of language models across multiple challenging benchmarks. It measures whether models can correctly solve math problems and follow structured reasoning processes aligned with a teacher model's trace. Use when the user wants to benchmark on MATH-500, MINERVA, OlympiadBench, LiveMathBench, KSAT2025, AIME 2024, AIME 2025, or asks about evaluating this task. Reports Pass@1.

- Skill: `qhjqhj00/math-reasoning-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/math-reasoning-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/math-reasoning-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/math-reasoning-eval

---


# math-reasoning-eval

> Reasoning-Aware GRPO using Process Mining — Taekhyun Park, Yongjae Lee, Hyerim Bae (2025) (arXiv:2510.25065, 2025)

## What this evaluates

Evaluates the mathematical reasoning capabilities of language models across multiple challenging benchmarks. It measures whether models can correctly solve math problems and follow structured reasoning processes aligned with a teacher model's trace.

## Datasets

- **MATH-500** — total ?; splits: test (-1)
- **MINERVA** — total ?; splits: test (-1)
- **OlympiadBench** — total ?; splits: test (-1)
- **LiveMathBench** — total ?; splits: test (-1)
- **KSAT2025** — total ?; splits: test (-1)
- **AIME 2024** — total ?; splits: test (-1)
- **AIME 2025** — total ?; splits: test (-1)

## Metrics

- `Pass@1` **(primary)** — range: percent
  - Pass@1 measures the percentage of problems for which the first generated response exactly matches the ground truth answer after answer extraction.

## Input / output format

**Input**: Mathematical problem statements or prompts requiring step-by-step reasoning and a final answer.

**Output**: A reasoning trace followed by a final extracted answer.

## Scoring recipe

```python
correct = 0
for problem, gold in zip(problems, gold_answers):
    response = model.generate(problem, temperature=0.6, top_p=0.95, max_tokens=16384)
    pred = extract_answer(response, library='DeepMath Evaluation Library')
    if pred == gold:
        correct += 1
pass_at_1 = (correct / len(problems)) * 100
```

## Common pitfalls

- MATH-500 and AIME 2024 results are reported for reference only due to potential data contamination and should not be used for primary conclusions.
- Answer extraction must use the DeepMath Evaluation Library, as raw text matching will fail due to formatting variations in reasoning traces.

## Evidence (verbatim from paper)

> All evaluations used the Pass@1 metric temperature \= 0.6, top_p \= 0.95, and max_tokens \= 16384, following the flashinfer*(Ye et al., [2025])* framework.\nThe answers are extracted using the DeepMath Evaluation Library*(He et al., [2025b])*.

## Citation

```bibtex
@misc{park2025reasoningawaregrpo,
  title={Reasoning-Aware GRPO using Process Mining},
  author={Taekhyun Park, Yongjae Lee, Hyerim Bae (2025)},
  year={2025},
  note={arXiv:2510.25065}
}
```

- arXiv: 2510.25065