Brats2018 Eval

brats2018-eval

No New-Net — Isensee et al. (2018) (arXiv:1809.10483, 2018)

What this evaluates

Evaluates 3D medical image segmentation models on brain tumor subregions (enhancing tumor, whole tumor, tumor core) using multimodal MRI scans. It probes the model's ability to accurately delineate complex, irregular tumor boundaries and differentiate tumors from surrounding vasculature and edema.

Datasets

• BraTS 2018 — total 351; splits: train (285), val (66), test (-1)

Metrics

• Dice (primary) — range: [0, 1]
  • Dice coefficient computed per case for three regions: enhancing tumor, whole tumor, and tumor core. Results are averaged across all cases in the split.
• HD95 — range: other
  • 95th percentile of the Hausdorff distance between predicted and ground truth segmentation boundaries, computed per case for the same three regions.

Input / output format

Input: Multimodal 3D MRI volumes (e.g., FLAIR, T1ce) with corresponding ground truth segmentation masks for tumor subregions.

Output: 3D segmentation mask with voxel-wise labels for enhancing tumor, necrosis, and edema.

Scoring recipe

def compute_dice(pred, gold):
    intersection = np.sum(pred * gold)
    return 2.0 * intersection / (np.sum(pred) + np.sum(gold))

def compute_hd95(pred, gold):
    dists = hausdorff_distance(pred, gold)
    return np.percentile(dists, 95)

# Per case, then average across cases for each region

Common pitfalls

• Metrics are computed by the official online evaluation platform, not locally, so local implementations may differ slightly.
• Co-training data (e.g., Decathlon) may overlap with the BraTS validation split, potentially inflating cross-validation scores.
• Final test results use an ensemble of 10 models (2 main models x 5 CV folds), not a single model evaluation.

Evidence (verbatim from paper)

Metrics were computed by the online evaluation platform. ... With Dice scores of 73.43 / 89.76 / 82.17 (enh/whole/core) on the training set this baseline model is by itself already very strong, especially when compared to the model of Isensee et al. [6] that achieved the third place in BraTS2017 (the training data for both challenges is identical, allowing a direct comparison of the models).

Citation

@misc{isensee2018nonewnet,
  title={No New-Net},
  author={Isensee et al. (2018)},
  year={2018},
  note={arXiv:1809.10483}
}

• arXiv: 1809.10483