# Rm Bench Eval

> Evaluates reward models' ability to correctly identify preferred responses based on substantive content rather than superficial stylistic cues. It probes sensitivity to subtle correctness differences, resistance to verbosity/style bias, and performance across diverse domains like math, code, and safety. Use when the user wants to benchmark on RM-Bench, or asks about evaluating this task. Reports Average Accuracy.

- Skill: `qhjqhj00/rm-bench-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/rm-bench-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/rm-bench-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/rm-bench-eval

---


# rm-bench-eval

> RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style — Liu et al. (2024) (arXiv:2410.16184, 2024)

## What this evaluates

Evaluates reward models' ability to correctly identify preferred responses based on substantive content rather than superficial stylistic cues. It probes sensitivity to subtle correctness differences, resistance to verbosity/style bias, and performance across diverse domains like math, code, and safety.

## Datasets

- **RM-Bench** — total ?; splits: test (-1); repo https://github.com/THU-KEG/RM-Bench

## Metrics

- `Average Accuracy` **(primary)** — range: percent
  - Standard classification accuracy: the proportion of correctly predicted preferences out of total pairwise comparisons. Reported as Average Accuracy, Easy Accuracy, Normal Accuracy, and Hard Accuracy across domains.

## Input / output format

**Input**: A prompt paired with two candidate responses (a chosen response and a rejected response), often manipulated for style (e.g., verbose vs. concise) or difficulty level.

**Output**: A binary preference prediction indicating which response is preferred, or a scalar reward score used to derive the preference.

## Scoring recipe

```python
correct = 0
total = 0
for prompt, chosen, rejected in dataset:
    pred = model.predict(prompt, chosen, rejected)
    if pred == chosen:
        correct += 1
    total += 1
accuracy = (correct / total) * 100
```

## Common pitfalls

- Models often exhibit strong style bias, preferring verbose or well-structured responses over substantively correct ones, leading to Hard Accuracy dropping below the 50% random baseline.
- Performance varies drastically by domain; Math and Code tasks consistently yield the lowest accuracy, often lagging behind random guessing.
- DPO-based reward models heavily depend on the availability of a reference model during inference; removing it causes significant performance degradation.

## Evidence (verbatim from paper)

> The performance of the top-20 reward models on RM-Bench is shown in Table[3]. As the table demonstrates: 1) RM-Bench is Challenging: Our experiments show that even state-of-the-art models, such as Skywork-Reward-Llama-3.1-8B, achieve only 70.1% Average Accuracy and 46.6% Hard Accuracy in RM-Bench. Compared to a random-guessing baseline (50%), the results are far from satisfactory, indicating significant room for improvement. 2) Style Bias is Serious: Hard Accuracy on RM-Bench is significantly lower than Normal Accuracy, with most reward models failing to exceed random-level performance (50%).

## Citation

```bibtex
@misc{liu2024rmbench,
  title={RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style},
  author={Liu et al. (2024)},
  year={2024},
  note={arXiv:2410.16184}
}
```

- arXiv: 2410.16184