# Mmoral Bench Eval

> Evaluates large vision-language models' ability to interpret panoramic dental X-rays. It probes fine-grained anatomical recognition, pathology detection, and clinical report generation across multiple question types. Use when the user wants to benchmark on MMOral-Bench, or asks about evaluating this task. Reports accuracy.

- Skill: `qhjqhj00/mmoral-bench-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/mmoral-bench-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/mmoral-bench-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/mmoral-bench-eval

---


# mmoral-bench-eval

> Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis — Jing Hao et al. (2025) (arXiv:2509.09254, 2025)

## What this evaluates

Evaluates large vision-language models' ability to interpret panoramic dental X-rays. It probes fine-grained anatomical recognition, pathology detection, and clinical report generation across multiple question types.

## Datasets

- **MMOral-Bench** — total 20563; splits: test (-1); repo https://github.com/isbrycee/OralGPT

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly answered instances out of the total number of instances in a given category (e.g., Teeth, Patho, Overall).

## Input / output format

**Input**: Panoramic X-ray image paired with a text question or instruction.

**Output**: Text response. Closed-ended questions expect short/selected answers; open-ended questions expect free-form text or clinical reports.

## Scoring recipe

```python
def calculate_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if normalize(p) == normalize(g))
    return (correct / len(gold)) * 100
```

## Common pitfalls

- Proprietary models frequently refuse to answer due to safety/ethical filters rather than inability.
- Open-ended VQA tasks consistently yield much lower scores than closed-ended tasks across nearly all models.
- Models show strong bias toward coarse anatomical structures (e.g., Jaw) while struggling with fine-grained details (e.g., Teeth, Pathology).

## Evidence (verbatim from paper)

> Current universal medical LVLMs achieve less than 40% average accuracy on MMOral-Bench, with HealthGPT-XL32 attaining peak performance at 39.59%.

## Citation

```bibtex
@misc{hao2025towardsbetterdentalai,
  title={Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis},
  author={Jing Hao et al. (2025)},
  year={2025},
  note={arXiv:2509.09254}
}
```

- arXiv: 2509.09254