qhjqhj00 auto-j-eval
Generative Judge for Evaluating Alignment — Li et al. (2023) (arXiv:2310.05470, 2023)
What this evaluates
Evaluates the alignment and preference alignment of LLMs through pairwise comparison, single-response critique generation, and overall rating. It probes the model's ability to consistently identify human-preferred responses, generate structured natural language critiques, and rank outputs according to a reference judge (GPT-4).
Datasets
- Eval-P — total ?; splits: test (-1)
- Eval-C — total ?; splits: test (-1)
- Eval-R — total 1993; splits: test (1993)
- AlpacaEval — total ?; splits: test (-1)
Metrics
agreement rate(primary) — range: [0, 1]- Percentage of test cases where the model's judgment matches the human judgment, requiring consistent results when the two response positions are swapped.
Spearman correlation— range: [-1, 1]- Rank-based correlation coefficient measuring the monotonic relationship between the model's predicted ratings and the reference GPT-4 ratings.
Input / output format
Input: Pairwise: Query string followed by two candidate responses (A and B). Single-response: Query string followed by one candidate response.
Output: Pairwise: Structured natural language critique followed by a final decision (e.g., 'Response 1' or 'Response 2'). Single-response: Structured natural language critique followed by a numerical rating in double brackets (e.g., '[[5]]').
Scoring recipe
def compute_agreement_rate(predictions, gold, swapped_predictions):
consistent = [p == sp for p, sp in zip(predictions, swapped_predictions)]
matches_human = [p == g for p, g in zip(predictions, gold)]
agreements = [c and m for c, m in zip(consistent, matches_human)]
return sum(agreements) / len(agreements)
def compute_spearman_corr(model_scores, gold_scores):
return spearmanr(model_scores, gold_scores).correlation
Common pitfalls
- Positional bias: Failing to swap response orders in the prompt leads to inconsistent judgments and inflated agreement rates.
- Tie handling: Reference judges like GPT-4 rarely output ties, while humans do frequently; this mismatch can skew critique comparison metrics.
- Consistency requirement: Agreement is only counted if the model's judgment matches the human label in BOTH the original and swapped response order prompts.
Evidence (verbatim from paper)
The agreement rates for AUTO-J and the baselines on Eval-P are in Tab. 1. AUTO-J achieves a significantly higher agreement rate than all baselines except GPT-4 on every scenario group. We also plot the prediction consistency for each model in Fig. 4. AUTO-J has a similar consistency rate to GPT-4 and is far more consistent than all other baselines, which makes it a more reliable and robust judge for pairwise comparison.
Citation
@misc{li2023generativejudge,
title={Generative Judge for Evaluating Alignment},
author={Li et al. (2023)},
year={2023},
note={arXiv:2310.05470}
}
- arXiv: 2310.05470