Brats2021 Seg Eval

brats2021-seg-eval

Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images — Hatamizadeh et al. (2022) (arXiv:2201.01266, 2022)

What this evaluates

Evaluates 3D semantic segmentation of brain tumor sub-regions on multi-modal MRI scans. It probes the model's ability to capture long-range spatial dependencies and multi-scale contextual information for precise tumor boundary delineation.

Datasets

• BraTS 2021 — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• Dice score (primary) — range: [0, 1]
  • Harmonic mean of precision and recall per voxel class, averaged across Enhancing Tumor (ET), Whole Tumor (WT), and Tumor Core (TC).
• Hausdorff distance — range: other
  • 95th percentile Hausdorff distance in millimeters, measuring the maximum boundary discrepancy between predicted and ground truth segmentations.

Input / output format

Input: Multi-modal 3D MRI volumes (FLAIR, T1, T1ce, T2) with corresponding voxel-wise segmentation masks for tumor sub-regions.

Output: 3D voxel-wise probability maps or binary masks for three tumor classes: Enhancing Tumor (ET), Whole Tumor (WT), and Tumor Core (TC).

Scoring recipe

def dice_score(pred, gt):
    intersection = np.sum(pred * gt)
    union = np.sum(pred) + np.sum(gt)
    return 2.0 * intersection / (union + 1e-6)

def hausdorff_95(pred, gt):
    dists = distance_transform_edt(~pred) + distance_transform_edt(~gt)
    return np.percentile(dists, 95)

# Average across ET, WT, TC classes
mean_dice = np.mean([dice_score(p, g) for p, g in zip(preds, gts)])
mean_hd = np.mean([hausdorff_95(p, g) for p, g in zip(preds, gts)])

Common pitfalls

• Dice score is class-averaged; reporting only one class (e.g., WT) overestimates overall performance.
• Hausdorff distance is highly sensitive to outliers; the paper reports mean values but does not explicitly state if 95% HD or full HD is used, though BraTS standard is 95% HD.
• Internal 5-fold CV uses a different split than the official BraTS 2021 test set, making direct comparison to challenge rankings tricky.

Evidence (verbatim from paper)

Evaluation results across all five folds are presented in Table 2. The proposed Swin UNETR model outperforms all competing approaches across all 5 folds and on average for all semantic classes (e.g. ET, WT, TC). Specifically, Swin UNETR outperforms the closest competing approaches by $0.7%,0.6%$ and $0.4%$ for ET,WT and TC classes respectively and on average $0.5%$ across all classes in all folds. ... Table 4: BraTS 2021 testing dataset benchmarks in terms of mean Dice score and Hausdorff distance values.

Citation

@misc{hatamizadeh2022swinunetr,
  title={Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images},
  author={Hatamizadeh et al. (2022)},
  year={2022},
  note={arXiv:2201.01266}
}

• arXiv: 2201.01266