Brats 2023 Meningioma Eval

brats-2023-meningioma-eval

Analysis of the BraTS 2023 Intracranial Meningioma Segmentation Challenge — LaBella et al. (2024) (arXiv:2405.09787, 2024)

What this evaluates

Evaluates 3D medical image segmentation models on intracranial meningioma MRI scans. It probes volumetric accuracy and boundary sharpness across three tumor subregions (enhancing tumor, tumor core, whole tumor) under varying contrast and lesion size conditions.

Datasets

• BraTS 2023 Intracranial Meningioma Challenge — total 1424; splits: train (1000), val (141), test (283)

Metrics

• DSC (primary) — range: [0, 1]
  • Dice Similarity Coefficient: 2 * |A ∩ B| / (|A| + |B|), where A and B are prediction and ground truth masks. Measures volumetric overlap.
• 95HD — range: other
  • 95th percentile of the Hausdorff Distance between prediction and ground truth surfaces. Measures boundary sharpness in millimeters.

Input / output format

Input: Multi-sequence 3D pre-operative meningioma MRI volumes.

Output: 3D segmentation masks for three regions: Enhancing Tumor (ET), Tumor Core (TC), and Whole Tumor (WT).

Scoring recipe

def compute_dsc(pred, gt):
    intersection = np.sum(pred * gt)
    return 2.0 * intersection / (np.sum(pred) + np.sum(gt))

def compute_95hd(pred, gt):
    # Compute surface distances between prediction and ground truth
    dists = surface_distances(pred, gt)
    return np.percentile(dists, 95)

# Aggregate per case, then average across test set
dsc_scores = [compute_dsc(p, g) for p, g in zip(predictions, ground_truth)]
hd_scores = [compute_95hd(p, g) for p, g in zip(predictions, ground_truth)]
final_dsc = np.mean(dsc_scores)
final_95hd = np.mean(hd_scores)

Common pitfalls

• Models often overfit to public validation metrics, leading to degraded test-phase performance.
• Performance significantly drops on heavily calcified lesions with low contrast or small volumes due to partial volume effects.
• DSC measures volumetric overlap but can mask poor boundary delineation; 95HD is required to assess lesion boundary sharpness.

Evidence (verbatim from paper)

A total of 1000 training (70%) multi-sequence pre-operative meningioma MRI cases, 141 validation cases (10%), and 283 test cases (20%) were utilized within the BraTS Meningioma Challenge (Table 1) in adherence with standard machine learning protocols. The statistical summary of the teams' performances is outlined in Tables 2 and 3; which list the calculated DSCs and 95HD, respectively.

Citation

@misc{labella2024analysis,
  title={Analysis of the BraTS 2023 Intracranial Meningioma Segmentation Challenge},
  author={LaBella et al. (2024)},
  year={2024},
  note={arXiv:2405.09787}
}

• arXiv: 2405.09787