Brats2021 Segmentation Eval

brats2021-segmentation-eval

BMDS-Net: A Bayesian Multi-Modal Deep Supervision Network for Robust Brain Tumor Segmentation — Zhou et al. (2026) (arXiv:2601.17504, 2026)

What this evaluates

Evaluates 3D brain tumor segmentation accuracy and robustness under missing MRI modalities using multi-modal MRI scans. It probes a model's ability to delineate tumor sub-regions while maintaining calibration and stability when contrast sequences are corrupted or absent.

Datasets

• BraTS 2021 — total 1251; splits: train (-1), val (-1), test (-1)

Metrics

• Dice Score (primary) — range: [0, 1]
  • Dice similarity coefficient computed per tumor sub-region (Whole Tumor, Tumor Core, Enhancing Tumor) between predicted and ground truth segmentation masks.
• HD95 — range: other
  • 95th percentile of the Hausdorff distance between predicted and ground truth boundaries, measured in millimeters.
• ECE — range: [0, 1]
  • Expected Calibration Error measuring the alignment between predicted confidence probabilities and actual accuracy.

Input / output format

Input: Multi-modal 3D MRI volumes (T1, T1ce, T2, FLAIR) preprocessed with z-score normalization and randomly cropped to 128x128x128 voxels during training.

Output: Voxel-wise segmentation masks for three tumor sub-regions (WT, TC, ET) and voxel-wise uncertainty estimates.

Scoring recipe

def compute_dice(pred, gt):
    intersection = np.sum(pred * gt)
    union = np.sum(pred) + np.sum(gt)
    return 2.0 * intersection / (union + 1e-5)

def compute_hd95(pred, gt, spacing=1.0):
    dist = hausdorff_distance(pred, gt)
    return np.percentile(dist, 95) * spacing

def compute_ece(confidence, accuracy, n_bins=15):
    bin_boundaries = np.linspace(0, 1, n_bins + 1)
    ece = 0.0
    for i in range(n_bins):
        mask = (confidence >= bin_boundaries[i]) & (confidence < bin_boundaries[i+1])
        if np.sum(mask) > 0:
            bin_acc = np.mean(accuracy[mask])
            bin_conf = np.mean(confidence[mask])
            ece += np.sum(mask) * abs(bin_acc - bin_conf)
    return ece / len(accuracy)

Common pitfalls

• The test set is extremely small (5% of 1,251 ≈ 62 scans), which can cause high variance in reported metrics and limit statistical power.
• Missing-modality robustness is evaluated on the validation set rather than the official BraTS test set, making direct comparison with public benchmarks difficult.
• HD95 is reported in millimeters, but the exact voxel spacing/resolution used for conversion is not explicitly stated, which affects cross-study comparability.

Evidence (verbatim from paper)

BMDS-Net achieves competitive Dice scores comparable to the highly optimized nnU-Net. Crucially, compared to the vanilla Swin UNETR, our method demonstrates consistent improvements across all tumor sub-regions, validating the efficacy of our proposed modules.

Citation

@misc{zhou2026bmdsnet,
  title={BMDS-Net: A Bayesian Multi-Modal Deep Supervision Network for Robust Brain Tumor Segmentation},
  author={Zhou et al. (2026)},
  year={2026},
  note={arXiv:2601.17504}
}

• arXiv: 2601.17504