# Math Reward Eval

> Evaluates multimodal and language-only models on mathematical reasoning, self-judgment/reward accuracy, and general multimodal capabilities. It measures how well a model can solve complex problems, verify its own answers, and generalize across diverse domains without external reward models. Use when the user wants to benchmark on MathVista, GSM8k, RewardBench2, VL-RewardBench, MMBench, MMStar, or asks about evaluating this task. Reports accuracy.

- Skill: `qhjqhj00/math-reward-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/math-reward-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/math-reward-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/math-reward-eval

---


# math-reward-eval

> SPARK: Synergistic Policy And Reward Co-Evolving Framework — Liu et al. (2025) (arXiv:2509.22624, 2025)

## What this evaluates

Evaluates multimodal and language-only models on mathematical reasoning, self-judgment/reward accuracy, and general multimodal capabilities. It measures how well a model can solve complex problems, verify its own answers, and generalize across diverse domains without external reward models.

## Datasets

- **MathVista** — total ?; splits: test (-1)
- **GSM8k** — total ?; splits: test (-1)
- **RewardBench2** — total ?; splits: test (-1)
- **VL-RewardBench** — total ?; splits: test (-1)
- **MMBench** — total ?; splits: test (-1)
- **MMStar** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of instances where the model's predicted answer or correctness judgment matches the ground truth. Computed as (correct predictions / total instances) * 100.

## Input / output format

**Input**: Text and/or image prompts containing mathematical problems or general multimodal questions.

**Output**: Final answer string or a binary correctness judgment (correct/incorrect).

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = 0
    for p, g in zip(predictions, gold):
        if normalize_answer(p) == normalize_answer(g):
            correct += 1
    return (correct / len(gold)) * 100
```

## Common pitfalls

- Reward and math benchmarks often require strict formatting for answer extraction; minor formatting differences (e.g., fractions vs decimals, extra whitespace) can cause false negatives if not normalized.
- Self-judgment accuracy evaluation requires the model to output both reasoning steps and a binary correctness label, which can be affected by chain-of-thought verbosity or inconsistent labeling formats.

## Evidence (verbatim from paper)

> We evaluate the model’s judgment ability by measuring its accuracy in determining whether its own answers are correct. Based on these judgments, we compute recall, precision, and F1 scores.

## Citation

```bibtex
@misc{liu2025spark,
  title={SPARK: Synergistic Policy And Reward Co-Evolving Framework},
  author={Liu et al. (2025)},
  year={2025},
  note={arXiv:2509.22624}
}
```

- arXiv: 2509.22624