Task01 Braintumour Eval

task01-braintumour-eval

A large annotated medical image dataset for the development and evaluation of segmentation algorithms — Amber L. Simpson et al. (arXiv:1902.09063, 2019)

What this evaluates

Evaluates a model's ability to perform 3D semantic segmentation on multi-parametric MRI scans of brain tumours. It probes the algorithm's capacity to distinguish and delineate multiple tumour sub-regions (edema, enhancing, non-enhancing) under high clinical variability in scanner hardware and acquisition protocols.

Datasets

• Task01_BrainTumour — total 750; splits: train (-1), test (-1)

Metrics

• semantic segmentation accuracy (primary) — range: [0, 1]
  • Standard volumetric overlap metric, typically Dice coefficient: 2|A∩B|/(|A|+|B|), where A and B are the predicted and ground-truth segmentation masks. Measures voxel-wise overlap between predicted and expert-annotated tumour sub-regions.

Input / output format

Input: 3D multi-parametric MRI volumes (T1, T1-Gd, T2, T2-FLAIR) in NIfTI format, co-registered to SRI24 atlas space, resampled to 1mm³ isotropic voxels, and skull-stripped.

Output: 3D segmentation mask in NIfTI format with discrete labels for each tumour sub-region (edema, enhancing, non-enhancing).

Scoring recipe

def compute_dice(pred_mask, gold_mask):
    intersection = np.sum(pred_mask * gold_mask)
    union = np.sum(pred_mask) + np.sum(gold_mask)
    return 2.0 * intersection / union if union > 0 else 0.0
# Compute per-sub-region, then average across sub-regions and subjects.

Common pitfalls

• Images are rotated 180 degrees in the axial plane to correct misformatted headers; models must assume RAS coordinate frame.
• Multi-modal input required (4 sequences per scan); failing to concatenate or process all modalities degrades performance.
• Highly variable acquisition protocols across 19 institutions and 1T-3T scanners; models trained on single-site data often fail to generalize.

Evidence (verbatim from paper)

The datasets were chosen based on availability and appropriateness for semantic segmentation algorithm development. Gold standard annotations for all tumour sub-regions in all scans were approved by expert board-certified neuradiologists.

Citation

@misc{simpson2019msd,
  title={A large annotated medical image dataset for the development and evaluation of segmentation algorithms},
  author={Amber L. Simpson et al.},
  year={2019},
  note={arXiv:1902.09063}
}

• arXiv: 1902.09063