Rationalrewards Eval

rationalrewards-eval

RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time — Haozhe Wang et al. (arXiv:2604.11626, 2026)

What this evaluates

Evaluates a reasoning-based reward model's ability to produce human-aligned preference judgments and optimize visual generation models via reinforcement learning and test-time prompt refinement.

Datasets

• Multimodal Reward Bench 2 (MMRB2) — total ?; splits: T2I (-1), Edit (-1)
• EditReward Bench — total ?; splits: Edit (-1)
• GenAI-Bench — total ?; splits: T2I (-1), Edit (-1)
• ImgEdit-Bench — total ?; splits: Edit (-1)
• GEdit-Bench-EN — total ?; splits: Edit (-1)
• UniGen (UniGenBench++) — total ?; splits: T2I (-1)
• PICA-Bench — total ?; splits: Edit (-1)

Metrics

• pairwise comparison accuracy (primary) — range: percent
  • Percentage of correctly predicted preference labels (or ranked order) compared to human annotations across benchmark splits.
• overall benchmark score — range: percent
  • Aggregated score across subcategories (e.g., Action, Attribute, Layout, etc.) reported per benchmark protocol.

Input / output format

Input: User prompt and generated image (for preference evaluation); user prompt and source image (for editing tasks).

Output: Structured multi-dimensional critiques with natural language refinement suggestions and dimension scores (threshold 3.0); or scalar/ranked preference labels for baseline comparison.

Scoring recipe

def compute_metrics(predictions, gold_labels, benchmark_protocol):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    accuracy = (correct / len(gold_labels)) * 100
    overall = sum(benchmark_protocol.compute_subcategory_scores(predictions)) / len(benchmark_protocol.subcategories)
    return {'pairwise_accuracy': accuracy, 'overall_score': overall}

Common pitfalls

• Confusing direct SFT distillation from a large teacher with the PARROT preference-anchored rationalization pipeline.
• Assuming test-time prompt tuning replaces RL entirely, whereas the paper shows they are complementary and prompt tuning often matches or exceeds RL only after RL or on specific benchmarks.
• Overlooking that training data uses raw preference pairs without explanations, requiring the PARROT pipeline to generate rationales before evaluation.

Evidence (verbatim from paper)

We report pairwise comparison accuracy on three established benchmarks: Multimodal Reward Bench 2 (Hu et al., 2025) and GenAI-Bench (Jiang et al., 2024) and EditReward Bench (Wu et al., 2025e) for both text- and image-to-image generation.

Citation

@misc{wang2026rationalrewards,
  title={RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time},
  author={Haozhe Wang et al.},
  year={2026},
  note={arXiv:2604.11626}
}

• arXiv: 2604.11626