Brats 2017 Eval

brats-2017-eval

Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge — Isensee et al. (2018) (arXiv:1802.10508, 2018)

What this evaluates

Evaluates 3D brain tumor segmentation accuracy across three sub-regions (whole tumor, core, enhancing) and tests radiomics-based survival prediction performance on multi-modal MRI scans.

Datasets

• BraTS 2017 — total ?; splits: train (285), val (46), test (-1)

Metrics

• Dice score (primary) — range: [0, 1]
  • Dice = 2 * |A ∩ B| / (|A| + |B|), measuring voxel-wise overlap between prediction and ground truth masks.
• RMSE — range: other
  • Root Mean Squared Error between predicted and actual survival times.
• Spearman correlation — range: [-1, 1]
  • Rank-based correlation coefficient between predicted and actual survival times.

Input / output format

Input: 3D multi-modal MRI volumes (T1, T1ce, T2, FLAIR) with corresponding voxel-wise segmentation masks and patient survival labels.

Output: Voxel-wise segmentation masks for three tumor sub-regions and a predicted survival time/value.

Scoring recipe

def dice_score(pred_mask, gold_mask):
    intersection = np.sum(pred_mask * gold_mask)
    return 2.0 * intersection / (np.sum(pred_mask) + np.sum(gold_mask))

def rmse(pred_times, gold_times):
    return np.sqrt(np.mean((pred_times - gold_times) ** 2))

Common pitfalls

• Cases with no enhancing tumor in the ground truth yield a Dice score of zero by definition, which can significantly lower the mean score.
• Manual ground truth annotations may contain errors, such as blood vessels being labeled as enhancing tumor, affecting segmentation metrics.
• Overfitting to the validation set is mitigated by limiting submissions, but test set performance may still drop due to difficult cases.

Evidence (verbatim from paper)

Quantitatively, we achieve Dice scores of 0.896, 0.797 and 0.732 for whole, core and enhancing, respectively, on the BraTS 2017 validation set.

Citation

@misc{isensee2018brats,
  title={Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge},
  author={Isensee et al. (2018)},
  year={2018},
  note={arXiv:1802.10508}
}

• arXiv: 1802.10508