Multimodal Reward Benchmarks Eval

multimodal-reward-benchmarks-eval

You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass — Yang et al. (2026) (arXiv:2604.10966, 2026)

What this evaluates

Evaluates multimodal reward models on preference ranking tasks across image and video domains, measuring how well they score or rank candidate responses compared to ground-truth preferences. It compares multi-response scoring against single-response baselines and generative judges, while also assessing inference efficiency and downstream policy optimization stability.

Datasets

• VL-RewardBench — total ?; splits: test (-1)
• Multimodal RewardBench — total ?; splits: test (-1)
• MM-RLHF RewardBench — total ?; splits: test (-1)
• MR2Bench-Image — total ?; splits: test (-1)
• VideoRewardBench — total ?; splits: test (-1)
• MR2Bench-Video — total ?; splits: test (-1)

Metrics

• pairwise accuracy (primary) — range: [0, 100] percent
  • Percentage of correctly predicted preferred responses in pairwise comparisons. Reported as macro pairwise accuracy for VL-RewardBench and VideoRewardBench, and standard pairwise accuracy for Multimodal RewardBench and MM-RLHF RewardBench.
• best-of-4 accuracy — range: [0, 100] percent
  • Percentage of samples where the model correctly selects the top-ranked response from a set of 4 candidates.

Input / output format

Input: Image or video context with N candidate responses concatenated using separator tokens.

Output: N scalar reward scores (one per response) or a predicted best response index.

Scoring recipe

def compute_accuracy(predictions, gold, metric_type):
    correct = 0
    for pred, gold_label in zip(predictions, gold):
        if metric_type == 'pairwise':
            if pred == gold_label: correct += 1
        elif metric_type == 'best-of-4':
            if pred == gold_label: correct += 1
    return (correct / len(gold)) * 100

Common pitfalls

• Pairwise accuracy and best-of-4 accuracy use different evaluation protocols and should not be averaged without noting the difference.
• Single-response Bradley-Terry scoring requires N forward passes per sample, while multi-response scoring uses one pass; comparing them requires equalizing compute or explicitly reporting the N× speedup.
• Response order bias can affect multi-response models if not shuffled during training or evaluated across permutations.

Evidence (verbatim from paper)

VL-RB: VL-RewardBench (macro pairwise acc.); MM-RB: Multimodal RewardBench (pairwise acc.); MMRLHF: MM-RLHF RewardBench (pairwise acc.); MR2B-I: MR2Bench-Image (best-of-4 acc.); VRB: VideoRewardBench (macro pairwise acc.); MR2B-V: MR2Bench-Video (best-of-4 acc.).

Citation

@misc{yang2026youonlyjudgeonce,
  title={You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass},
  author={Yang et al. (2026)},
  year={2026},
  note={arXiv:2604.10966}
}

• arXiv: 2604.10966