Llmbar Eval

llmbar-eval

Mitigating the Bias of Large Language Model Evaluation — Hongli Zhou et al. (arXiv:2409.16788, 2024)

What this evaluates

This benchmark probes an evaluator model's ability to accurately judge preference between two model responses while resisting bias towards superficial qualities like verbosity, fluency, and formality. It measures instruction-following accuracy by comparing judgments on natural preference data against adversarially crafted instances designed to confound less capable judges.

Datasets

• LLMBar — total ?; splits: Natural (-1), Neighbor (-1), GPTInst (-1), GPTOut (-1), Manual (-1)

Metrics

• accuracy (primary) — range: percent
  • Percentage of correctly judged preference pairs. Calculated as (number of instances where the model's judgment matches the ground-truth annotation) / (total number of instances) * 100.

Input / output format

Input: Instruction/prompt, Response A, Response B, and ground-truth preference label.

Output: Binary preference label indicating which response is preferred.

Scoring recipe

correct = 0
for pred, gold in zip(predictions, gold_labels):
    if pred == gold:
        correct += 1
accuracy = (correct / len(gold_labels)) * 100

Common pitfalls

• Evaluators often conflate superficial qualities (verbosity, fluency, formality) with instruction-following accuracy, leading to inflated scores on natural sets but poor performance on adversarial sets.
• The benchmark requires careful separation of 'Natural' and 'Adversarial' set results; reporting only the average masks the specific bias mitigation effects targeted by the paper.

Evidence (verbatim from paper)

Our experiments are based on LLMBar [[Zeng et al., 2023]], which consists a Natural set and four Adversarial sets. The Natural set collects and filters preference data from existing benchmarks, aiming to gauge evaluator performance in real-world distributions. Conversely, the Adversarial set comprises adversarially crafted instances that tend to confound less adept evaluators. As we can see, at the beginning of bias mitigation, the accuracy on the Natural set keeps stable, while the accuracy on the Adversarial set increase gradually.

Citation

@misc{zhou2024mitigatingbias,
  title={Mitigating the Bias of Large Language Model Evaluation},
  author={Hongli Zhou et al.},
  year={2024},
  note={arXiv:2409.16788}
}

• arXiv: 2409.16788