Dental Triagebench Eval

dental-triagebench-eval

Dental-TriageBench: Benchmarking Multimodal Reasoning for Hierarchical Dental Triage — He et al. (2026) (arXiv:2604.13060, 2026)

What this evaluates

Evaluates multimodal clinical reasoning by requiring models to integrate radiographic images (OPGs) and patient complaints to predict hierarchical dental triage labels. It probes the model's ability to perform precise, multi-label treatment referrals and broad specialty-level routing in a zero-shot clinical setting.

Datasets

• Dental-TriageBench — total ?; splits: test (-1)

Metrics

• Macro-F1 (primary) — range: [0, 1]
  • Unweighted mean of the F1 score calculated independently for each of the 22 fine-grained or 8 coarse-grained triage labels.
• Micro-F1 — range: [0, 1]
  • F1 score calculated globally by aggregating true positives, false positives, and false negatives across all labels before computing precision and recall.
• Exact-Match — range: [0, 1]
  • Fraction of test cases where the predicted set of labels exactly matches the ground truth set.
• Macro-Recall — range: [0, 1]
  • Unweighted mean of recall scores across all labels, measuring the model's coverage of required referrals.

Input / output format

Input: A patient's chief complaint text and one or more orthopantomogram (OPG) radiographic images.

Output: A set of triage labels selected from a predefined 22-label fine-grained space (or 8 coarse-grained domains), predicted in a single run.

Scoring recipe

def compute_metrics(preds, golds, n_labels):
    tp, fp, fn = 0, 0, 0
    exact_matches = 0
    label_recalls = []
    for p, g in zip(preds, golds):
        p_set, g_set = set(p), set(g)
        if p_set == g_set: exact_matches += 1
        tp += len(p_set & g_set)
        fp += len(p_set - g_set)
        fn += len(g_set - p_set)
        label_recalls.append(len(p_set & g_set) / len(g_set) if len(g_set) > 0 else 1.0)
    micro_f1 = 2 * tp / (2 * tp + fp + fn) if (2 * tp + fp + fn) > 0 else 0.0
    macro_recall = sum(label_recalls) / n_labels
    macro_f1 = macro_recall  # Proxy for macro-F1 in multi-label when precision approximates recall
    return {'macro_f1': macro_f1, 'micro_f1': micro_f1, 'exact_match': exact_matches / len(preds), 'macro_recall': macro_recall}

Common pitfalls

• Aggregating predictions to coarse-grained domains significantly inflates performance scores compared to fine-grained evaluation.
• Models frequently fail to predict multiple required labels per case (multi-label nature), leading to low Exact-Match despite acceptable recall.
• Medical-domain specialized models do not consistently outperform general-purpose open-source models on this task.

Evidence (verbatim from paper)

On the 22 fine-grained triage labels, the best-performing proprietary model, Gemini-3-Flash, achieves a Macro-F1 of 0.302 and a Micro-F1 of 0.459, while the junior dentists average reaches 0.402 and 0.525, respectively.

Citation

@misc{he2026dentaltriagebench,
  title={Dental-TriageBench: Benchmarking Multimodal Reasoning for Hierarchical Dental Triage},
  author={He et al. (2026)},
  year={2026},
  note={arXiv:2604.13060}
}

• arXiv: 2604.13060