Brats2017 Eval

brats2017-eval

Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks — Wang et al. (2017) (arXiv:1709.00382, 2017)

What this evaluates

Probes 3D brain tumor segmentation capability across multiple MRI sequences, evaluating boundary accuracy and volumetric overlap for hierarchical tumor subregions.

Datasets

• BraTS 2017 — total 477; splits: train (285), val (46), test (146)

Metrics

• Dice score (primary) — range: [0, 1]
  • 2 * |A ∩ B| / (|A| + |B|), measuring volumetric overlap between predicted and ground truth segmentation masks.
• Hausdorff distance — range: mm
  • Maximum of the distances from any point in one set to the closest point in the other set, measuring boundary discrepancy in millimeters.

Input / output format

Input: Four co-registered MRI sequences (T1, T1c, T2, FLAIR) per patient, skull-stripped and resampled to isotropic 1mm³ resolution.

Output: Three binary segmentation masks per patient corresponding to whole tumor (WT), tumor core (TC), and enhancing tumor core (ET).

Scoring recipe

def dice_score(pred, gt):
    intersection = np.sum(pred * gt)
    return 2.0 * intersection / (np.sum(pred) + np.sum(gt))

def hausdorff_distance(pred, gt):
    from scipy.ndimage import distance_transform_edt
    dist_pred = distance_transform_edt(1 - pred)
    dist_gt = distance_transform_edt(1 - gt)
    hd = max(np.max(dist_pred[gt == 1]), np.max(dist_gt[pred == 1]))
    return hd

Common pitfalls

• Metrics are computed by the official BraTS server, not locally.
• Hausdorff distance is highly sensitive to outliers, leading to high standard deviations.
• Subregions are hierarchical: ET ⊂ TC ⊂ WT.

Evidence (verbatim from paper)

We uploaded the segmentation results obtained by the experimental algorithms to the BraTS 2017 server, and the server provided quantitative evaluations including Dice score and Hausdorff distance compared with the ground truth.

Citation

@misc{wang2017automatic,
  title={Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks},
  author={Wang et al. (2017)},
  year={2017},
  note={arXiv:1709.00382}
}

• arXiv: 1709.00382