Math Reasoning Diversity Eval

math-reasoning-diversity-eval

Diversity-Aware Policy Optimization for Large Language Model Reasoning — Jian Yao et al. (2025) (arXiv:2505.23433, 2025)

What this evaluates

This evaluation protocol probes the mathematical reasoning capabilities and solution diversity of large language models. It measures how well models solve grade-school and college-level math problems, and whether reinforcement learning fine-tuning preserves or degrades the variety of generated solution paths.

Datasets

• GSM8K — total ?; splits: train (-1), test (1319); HF gsm8k
• MATH500 — total ?; splits: test (500); HF hendrycks2021measuring
• Olympiad Bench — total ?; splits: test (-1); HF he2024olympiadbench
• College Math — total ?; splits: test (-1); HF tang2024mathscale

Metrics

• Pass@1 accuracy (primary) — range: percent
  • Percentage of questions where the single generated response exactly matches the ground truth answer.
• Avg@8 accuracy — range: percent
  • Average accuracy computed by generating 8 responses per question and checking if any matches the ground truth, averaged across all questions.
• Div-Equ — range: percent
  • Proportion of distinct equations or solution paths across generated responses.
• Div-N-gram — range: percent
  • Proportion of distinct n-grams per response, capturing intra-response diversity.
• Div-Self-BLEU — range: percent
  • 100 minus the Self-BLEU score across generated responses, capturing inter-response diversity.

Input / output format

Input: Mathematical word problems or equations presented as text prompts.

Output: Natural language reasoning steps followed by a final numerical or symbolic answer.

Scoring recipe

def compute_pass_at_k(predictions, gold_answers, k=1):
    correct = 0
    for i in range(0, len(predictions), k):
        batch = predictions[i:i+k]
        gold = gold_answers[i // k]
        if any(extract_final_answer(p) == gold for p in batch):
            correct += 1
    return (correct / len(gold_answers)) * 100

# For Avg@8, set k=8. For Pass@1, set k=1.
# Diversity metrics are computed over the set of generated responses per question.

Common pitfalls

• Using benchmarks with very few test items (e.g., AIME24 with 30 questions) leads to unstable and biased evaluation outcomes.
• Direct performance comparisons across baselines are confounded by differing computational resources and training datasets used during RL fine-tuning.
• Diversity metrics like Self-BLEU and N-gram diversity can be sensitive to tokenization choices and response formatting conventions.

Evidence (verbatim from paper)

We evaluate the reasoning performance using Pass@1 accuracy, as shown in Table[1]. We also report the performance against training steps in Figure [4] against the training steps ‣ Appendix E More experiment results ‣ Diversity-Aware Policy Optimization for Large Language Model Reasoning") in Appendix. In our experiment, R1-zero-Div demonstrates superior performance compared to R1-zero, achieving an average improvement of $3.5%$.

Citation

@misc{yao2025diversityaware,
  title={Diversity-Aware Policy Optimization for Large Language Model Reasoning},
  author={Jian Yao et al. (2025)},
  year={2025},
  note={arXiv:2505.23433}
}

• arXiv: 2505.23433