Meissa Medical Eval

meissa-medical-eval

Meissa: Multi-modal Medical Agentic Intelligence — Chen et al. (2026) (arXiv:2603.09018, 2026)

What this evaluates

Evaluates a 4B multi-modal medical agentic model's ability to perform clinical reasoning, tool use, and multi-step interaction across radiology, pathology, and clinical domains. Probes strategy selection (when to use tools vs direct reasoning) and execution policy under various agent frameworks.

Datasets

• MIMIC-CXR-VQA — total ?; splits: test (-1)
• ChestAgentBench — total ?; splits: test (-1)
• PathVQA — total ?; splits: test (-1)
• SLAKE — total ?; splits: test (-1)
• VQA-RAD — total ?; splits: test (-1)
• OmniMed — total ?; splits: test (-1)
• MedXpertQA — total ?; splits: test (-1)
• MedQA — total ?; splits: test (-1)
• PubMedQA — total ?; splits: test (-1)
• NEJM — total ?; splits: test (-1)
• NEJM Ext. — total ?; splits: test (-1)
• MIMIC-IV — total ?; splits: test (-1)
• MedQA Ext. — total ?; splits: test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • Exact-match accuracy for close-ended questions; soft-match accuracy with medical synonym normalization for open-ended VQA.

Input / output format

Input: Multi-modal medical queries (text, images like CXR/pathology/EHR, or text-only) with varying complexity, often requiring tool use or multi-turn interaction.

Output: Final answer or diagnosis, formatted as a direct response or multi-step agentic trajectory ending in a conclusion.

Scoring recipe

def compute_accuracy(predictions, gold):
    correct = 0
    for pred, gold in zip(predictions, gold):
        if is_close_ended(gold):
            if normalize(pred) == normalize(gold):
                correct += 1
        else:
            if medical_synonym_match(pred, gold):
                correct += 1
    return correct / len(gold)

Common pitfalls

• Decontamination only checks question text n-grams (n=8), not image-level leakage, though disjoint DICOM/hashing is used.
• Soft-match accuracy relies on medical synonym normalization, which may vary across benchmarks.
• Results are averaged over 3 seeds; variance is only in supplementary.

Evidence (verbatim from paper)

All tasks are formulated as QA or VQA; the evaluation metric is accuracy throughout: exact-match accuracy for close-ended questions and soft-match accuracy with medical synonym normalization*[aasmimic]* for open-ended VQA.

Citation

@misc{chen2026meissa,
  title={Meissa: Multi-modal Medical Agentic Intelligence},
  author={Chen et al. (2026)},
  year={2026},
  note={arXiv:2603.09018}
}

• arXiv: 2603.09018