Brats2023 Segmentation Eval

brats2023-segmentation-eval

AI-Driven MRI-based Brain Tumour Segmentation Benchmarking — Ludwig et al. (2025) (arXiv:2506.20786, 2025)

What this evaluates

Evaluates the zero-shot and fine-tuned performance of promptable and non-promptable 3D medical image segmentation models on brain tumor MRI data. It probes how prompt type (points vs. bounding boxes) and prompt accuracy affect segmentation quality compared to a strong unprompted baseline.

Datasets

• BraTS 2023 Adult Glioma — total ?; splits: train (-1)
• BraTS 2023 Pediatrics — total ?; splits: train (-1), test (-1)

Metrics

• Dice score (DSC) (primary) — range: [0, 1]
  • Dice Similarity Coefficient measuring overlap between predicted and ground truth segmentation masks: 2 * |prediction ∩ ground_truth| / (|prediction| + |ground_truth|). Reported as a value between 0 and 1.

Input / output format

Input: 3D MRI brain volumes with optional prompts (1, 5, or 10 points; or high/medium/low quality bounding boxes) indicating tumor location.

Output: 3D binary segmentation mask of the brain tumor region.

Scoring recipe

def compute_dice(pred_mask, gt_mask):
    intersection = np.sum(pred_mask & gt_mask)
    union = np.sum(pred_mask) + np.sum(gt_mask)
    if union == 0:
        return 1.0
    return 2.0 * intersection / union

Common pitfalls

• Zero-shot evaluation on the adult glioma dataset suffers from data leakage, inflating scores.
• Prompt quality is subjective; 'high/medium/low' box accuracy and point count significantly impact results, making zero-shot prompting impractical without precise ground-truth prompts.
• Video segmentation mode consistently underperforms image mode for SAM 2, but is often reported without distinction.

Evidence (verbatim from paper)

Table I shows the average Dice scores obtained by all models across all applicable prompting methods. Of these results, MedSAM, SAM-Med-3D and nnU-net all benefit from data leakage to varying degrees. nnU-Net has the highest Dice score across all models and prompting styles with a score of 0.958 despite being unprompted.

Citation

@misc{ludwig2025ai,
  title={AI-Driven MRI-based Brain Tumour Segmentation Benchmarking},
  author={Ludwig et al. (2025)},
  year={2025},
  note={arXiv:2506.20786}
}

• arXiv: 2506.20786