Sparse Rl Eval

sparse-rl-eval

Sparse-RL: Breaking the Memory Wall in LLM Reinforcement Learning via Stable Sparse Rollouts — Luo et al. (2026) (arXiv:2601.10079, 2026)

What this evaluates

This evaluation protocol assesses the mathematical reasoning capabilities of LLMs trained with sparse reinforcement learning under strict memory constraints. It measures how well models maintain accuracy on standard math benchmarks when policy rollouts are generated using compressed KV caches instead of full context.

Datasets

• GSM8K — total ?; splits: test (-1)
• MATH500 — total ?; splits: test (-1)
• Gaokao — total ?; splits: test (-1)
• Minerva Math — total ?; splits: test (-1)
• OlympiadBench — total ?; splits: test (-1)
• AIME24 — total ?; splits: test (-1); HF AI-MO/aimo-validation-aime
• AMC23 — total ?; splits: test (-1); HF AI-MO/aimo-validation-amc

Metrics

• Pass@1 / Avg@32 accuracy (primary) — range: [0, 1]
  • Pass@1: binary accuracy (1 if the single generated response matches the gold answer, 0 otherwise), averaged over the dataset. Avg@32: for each problem, sample 32 responses, compute the mean accuracy across the 32 samples, then average these means across all problems in the dataset.

Input / output format

Input: Mathematical reasoning problem prompts.

Output: Text response containing reasoning steps and a final answer.

Scoring recipe

def compute_metric(predictions, golds, metric_type):
    scores = []
    for pred, gold in zip(predictions, golds):
        if metric_type == 'pass1':
            scores.append(1.0 if pred.strip() == gold.strip() else 0.0)
        elif metric_type == 'avg32':
            scores.append(sum(1.0 if p.strip() == gold.strip() else 0.0 for p in pred) / 32.0)
    return sum(scores) / len(scores)

Common pitfalls

• For AIME24 and AMC23, failing to sample 32 responses per problem and average them will drastically underestimate performance compared to the reported Avg@32 metric.
• Confusing the training KV cache budget (512 tokens) with inference constraints; the bonus evaluation explicitly enforces the same R-KV compression during inference, which shifts the distribution if not replicated.
• Using dense rollout evaluation for the main Sparse-RL results instead of sparse rollout evaluation, which would invalidate the memory-constrained performance claims.

Evidence (verbatim from paper)

We evaluate the performance of our method on 7 standard mathematical reasoning benchmarks: GSM8KCobbe et al. ([2021]), MATH500Hendrycks et al. ([2021]), GaokaoZhang et al. ([2023a]), Minerva MathLewkowycz et al. ([2022]), OlympiadBenchHe et al. ([2024]), AIME24111huggingface.co/datasets/AI-MO/aimo-validation-aime and AMC23222huggingface.co/datasets/AI-MO/aimo-validation-amc. For AIME24 and AMC23, we report Avg@32: for each problem, we sample 32 responses and calculate the mean accuracy among these 32 responses; the final score is the average of the mean accuracy for each item over the dataset. For the other six benchmarks, we report Pass@1: generating one response per problem.

Citation

@misc{luo2026sparserl,
  title={Sparse-RL: Breaking the Memory Wall in LLM Reinforcement Learning via Stable Sparse Rollouts},
  author={Luo et al. (2026)},
  year={2026},
  note={arXiv:2601.10079}
}

• arXiv: 2601.10079