Medhal Eval

medhal-eval

MedHal: An Evaluation Dataset for Medical Hallucination Detection — Mehenni et al. (2025) (arXiv:2504.08596, 2025)

What this evaluates

Evaluates AI models' ability to detect factual inconsistencies (hallucinations) in medical text and generate grounded explanations for why statements are non-factual. It probes domain-specific factual consistency reasoning and binary classification under clinical constraints.

Datasets

• MedHal — total ?; splits: train (-1), test (-1)
• MedNLI — total ?; splits: test (-1)
• Hegselmann et al. (2024a) Hallucination Dataset — total 210; splits: test (210)

Metrics

• F1-score (primary) — range: [0, 1]
  • Harmonic mean of precision and recall: 2 * (P * R) / (P + R). Measures overall accuracy in distinguishing factual from non-factual medical statements.
• Precision, Recall — range: [0, 1]
  • Precision: TP / (TP + FP). Recall: TP / (TP + FN). Standard binary classification metrics for factuality detection.
• ROUGE-1, ROUGE-2, BLEU — range: [0, 1]
  • Standard n-gram overlap metrics (Lin 2004; Papineni et al. 2002) comparing generated explanations against ground truth. Only computed when both model prediction and gold label are non-factual.

Input / output format

Input: Medical text statements (from clinical notes, trials, QA, NLI, summarization, or IE tasks) provided within a prompt template asking the model to classify factual consistency and explain non-factual content.

Output: Binary classification label (factual or non-factual) followed by a textual explanation pinpointing the exact erroneous part of the statement if labeled non-factual.

Scoring recipe

def compute_metrics(preds, gold, pred_exps, gold_exps):
    tp = sum(1 for p, g in zip(preds, gold) if p == 'non-factual' and g == 'non-factual')
    fp = sum(1 for p, g in zip(preds, gold) if p == 'non-factual' and g == 'factual')
    fn = sum(1 for p, g in zip(preds, gold) if p == 'factual' and g == 'non-factual')
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
    
    valid_idx = [i for i, (p, g) in enumerate(zip(preds, gold)) if p == 'non-factual' and g == 'non-factual']
    r1 = rouge_score([gold_exps[i] for i in valid_idx], [pred_exps[i] for i in valid_idx], rouge_types=['rouge1'])
    r2 = rouge_score([gold_exps[i] for i in valid_idx], [pred_exps[i] for i in valid_idx], rouge_types=['rouge2'])
    bleu = bleu_score([gold_exps[i] for i in valid_idx], [pred_exps[i] for i in valid_idx])
    return f1, prec, rec, r1, r2, bleu

Common pitfalls

• Explanation metrics (ROUGE/BLEU) are only valid when both the model's prediction and the ground truth label indicate a non-factual statement; computing them on factual samples or mismatched predictions invalidates the metric.
• Downstream evaluation on MedNLI requires filtering out neutral-labeled samples, as the MedHal-trained models are not designed to detect neutral statements.
• Zero-shot prompting consistently fails on the Hegselmann et al. hallucination dataset; a 1-shot strategy is required for fair comparison.

Evidence (verbatim from paper)

We use two main types of metrics to evaluate the models: factuality metrics and explanation metrics. Factuality metrics measure how accurately a model identifies factual versus non-factual content. This includes common measures like precision, recall, and F1-score. Due to inconsistencies in model output when using the prompt format from Figure [5]. You can see these results in Table [6]. Explanation metrics assess the validity of the explanations that the models provide for non-factual statements. Specifically, these metrics check if a model, after identifying non-factual content, correctly pinpoints the exact erroneous part of the statement. The explanation metrics are ROUGE-1 (R1), ROUGE-2 (R2) Lin ([2004]), and BLEU Papineni et al. ([2002]) scores. To ensure a valid comparison, we only consider samples where both the model’s prediction and the ground truth label indicate a non-factual statement. This guarantees that a true explanation exists and that the model attempted to generate one.

Citation

@misc{mehenni2025medhal,
  title={MedHal: An Evaluation Dataset for Medical Hallucination Detection},
  author={Mehenni et al. (2025)},
  year={2025},
  note={arXiv:2504.08596}
}

• arXiv: 2504.08596