Ukbob Medical Segmentation Eval

ukbob-medical-segmentation-eval

UKBOB: One Billion MRI Labeled Masks for Generalizable 3D Medical Image Segmentation — Bourigault et al. (2025) (arXiv:2504.06908, 2025)

What this evaluates

Evaluates 3D medical image segmentation models on multi-modal MRI and CT scans across abdominal, brain, and whole-body anatomical domains. Probes the model's ability to produce accurate organ and tumor masks while measuring both volumetric overlap and boundary precision under zero-shot and fine-tuning settings.

Datasets

• AMOS — total 100; splits: train (50), test (50)
• BTCV — total 50; splits: train (30), test (20)
• BRATS — total 5880; splits: test (-1)
• UKBOB — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• Dice Score (primary) — range: [0, 1]
  • Measures volumetric overlap between predicted and ground truth masks: 2 * |A ∩ B| / (|A| + |B|). Reported as mean across classes.
• Hausdorff Distance — range: other
  • Assesses boundary discrepancy by computing the maximum distance between any point on the predicted boundary and the closest point on the ground truth boundary. Reported as mean across classes.

Input / output format

Input: 3D MRI or CT volumes, typically cropped to 96×96×96 voxels during training. Multi-modal intensity values are used as input channels.

Output: 3D segmentation masks (binary or multi-class) corresponding to target organs or tumors, matching the spatial dimensions of the input volume.

Scoring recipe

def compute_dice(pred, gt):
    intersection = np.sum(pred * gt)
    return 2.0 * intersection / (np.sum(pred) + np.sum(gt))

def compute_hausdorff(pred, gt):
    pred_boundary = find_boundaries(pred)
    gt_boundary = find_boundaries(gt)
    dists = cdist(pred_boundary, gt_boundary)
    return np.max(dists)

# Aggregate across classes and subjects
dice_scores = [compute_dice(p, g) for p, g in zip(predictions, ground_truth)]
hd_scores = [compute_hausdorff(p, g) for p, g in zip(predictions, ground_truth)]
mean_dice = np.mean(dice_scores)
mean_hd = np.mean(hd_scores)

Common pitfalls

• Mean Dice Score can be heavily skewed by class imbalance; reporting per-class scores or weighted averages is often necessary for fair comparison.
• Hausdorff Distance is extremely sensitive to small boundary noise or outliers, which can inflate scores even when overall overlap is high.
• Datasets (AMOS, BTCV, BRATS) use different voxel spacings and coordinate conventions, requiring consistent resampling and orientation normalization before evaluation.

Evidence (verbatim from paper)

We evaluate our model using the Dice Score and the Hausdorff Distance Metric, which are widely used in medical image segmentation [[43], [34]]. The Dice Score measures the overlap between predicted and ground truth masks, while the Hausdorff Distance assesses the boundary discrepancy, providing a comprehensive evaluation of segmentation performance.

Citation

@misc{bourigault2025ukbob,
  title={UKBOB: One Billion MRI Labeled Masks for Generalizable 3D Medical Image Segmentation},
  author={Bourigault et al. (2025)},
  year={2025},
  note={arXiv:2504.06908}
}

• arXiv: 2504.06908