Umd 3d Medical Seg Eval

umd-3d-medical-seg-eval

Uncovering Modality Discrepancy and Generalization Illusion for General-Purpose 3D Medical Segmentation — Zhang et al. (2026) (arXiv:2602.07643, 2026)

What this evaluates

Evaluates the cross-modality generalization and robustness of 3D medical segmentation foundation models by testing their ability to segment 13 whole-body organs in functional (PET) versus structural (CT/MRI) imaging using intrinsically paired intra-subject scans.

Datasets

• UMD Benchmark — total 954; splits: test (954); repo https://github.com/YichiZhang98/UMD

Metrics

• Dice Similarity Coefficient (DSC) (primary) — range: [0, 1]
  • Standard voxel-wise overlap metric for medical segmentation: DSC = 2 * |A ∩ B| / (|A| + |B|), where A and B are prediction and ground truth masks. Reported per organ and averaged across the 13 target organs.

Input / output format

Input: 3D volumetric medical images (whole-body PET, CT, or MRI scans) acquired from the same subject during a single diagnostic session.

Output: Voxel-wise segmentation masks identifying 13 target organs: liver, left kidney, right kidney, brain, heart, spleen, aorta, lung, colon, urinary bladder, pancreas, esophagus, and stomach.

Scoring recipe

def compute_dice(pred_mask, gt_mask):
    intersection = np.sum(pred_mask & gt_mask)
    union = np.sum(pred_mask) + np.sum(gt_mask)
    return 2.0 * intersection / union if union > 0 else 0.0

dsc_scores = [compute_dice(pred, gt) for pred, gt in zip(predictions, ground_truth)]
mean_dsc = np.mean(dsc_scores)

Common pitfalls

• Assuming performance on structural modalities (CT/MRI) generalizes to functional modalities (PET), which the benchmark explicitly shows fails due to low-contrast metabolic signals.
• Using recompiled public datasets that risk data leakage, whereas this benchmark uses newly collected intra-subject paired scans to guarantee evaluation validity.
• Entangling task complexity with modality differences, obscuring true cross-modality robustness.

Evidence (verbatim from paper)

Voxel-wise annotations were provided for 13 diverse organs, including the liver, left kidney, right kidney, brain, heart, spleen, aorta, lung, colon, urinary bladder, pancreas, esophagus, and stomach. By providing ground-truth labels for both structural and functional imaging, our protocol enables a head-to-head comparison between different modalities.

Citation

@misc{zhang2026umd,
  title={Uncovering Modality Discrepancy and Generalization Illusion for General-Purpose 3D Medical Segmentation},
  author={Zhang et al. (2026)},
  year={2026},
  note={arXiv:2602.07643}
}

• arXiv: 2602.07643