Brats2023 Eval

brats2023-eval

Ensemble Learning and 3D Pix2Pix for Comprehensive Brain Tumor Analysis in Multimodal MRI — Zeineldin et al. (2024) (arXiv:2412.11849, 2024)

What this evaluates

Evaluates 3D brain tumor segmentation and inpainting on multi-modal MRI scans. It probes the model's ability to accurately delineate tumor sub-regions (enhancing, core, whole) and synthesize realistic healthy tissue to replace tumor areas.

Datasets

• BraTS 2023 — total ?; splits: validation (-1), test (-1)

Metrics

• Lesion-wise DSC (primary) — range: [0, 1]
  • Measures overlap accuracy between predicted and ground truth masks. Lesion-wise calculation assigns a 0 score for false positives/negatives before averaging across cases.
• HD95 — range: other
  • 95th percentile Hausdorff Distance assessing the maximum deviation between predicted and actual segmentation boundaries.
• SSIM — range: [0, 1]
  • Structural Similarity Index Measure focusing on the realism of synthesized versus actual healthy tissue regions.
• PSNR — range: other
  • Peak Signal-to-Noise Ratio used to verify improvements in inpainting quality.
• MSE — range: other
  • Mean Square Error measuring the difference between synthesized and actual tissue regions.

Input / output format

Input: Multi-modal MRI volumes (e.g., T1, T2) with ground truth tumor masks and healthy tissue regions.

Output: Predicted 3D segmentation masks for tumor sub-regions (ET, TC, WT) and inpainted MRI volumes.

Scoring recipe

def lesion_wise_dsc(pred, gt):
    if pred.sum() == 0 and gt.sum() == 0: return 1.0
    if pred.sum() == 0 or gt.sum() == 0: return 0.0
    return 2.0 * (pred & gt).sum() / (pred.sum() + gt.sum())

def compute_metrics(preds, gts):
    case_scores = []
    for case in preds:
        lesion_scores = [lesion_wise_dsc(p, g) for p, g in zip(case['preds'], case['gts'])]
        case_scores.append(sum(lesion_scores) / len(lesion_scores))
    return sum(case_scores) / len(case_scores)

Common pitfalls

• Confusing lesion-wise averaging with case-wise averaging, which can hide poor performance on smaller lesions.
• Misclassifying Edema as Non-enhancing Tumor (NC) directly penalizes the Tumor Core (TC) metric due to overlapping label definitions.
• HD95 and DSC use fixed penalty values (0 and 374) for false positives/negatives before averaging, differing from standard continuous boundary metrics.

Evidence (verbatim from paper)

Our submissions to the BraTS challenge were evaluated based on lesion-wise Dice Similarity Coefficients (DSC) scores and the 95th percentile Hausdorff Distance (HD95), applied to whole, core, and active tumor regions. The DSC score measures the accuracy of our predicted segmentations against the ground truth, while the HD95 assesses the maximum deviation between predicted and actual segmentations. Lesionwise calculations penalize false positives and negatives by assigning a 0 Dice score and 374 for HD95, before averaging these scores for each case ID, ensuring a detailed assessment of segmentation precision.

Citation

@misc{zeineldin2024ensemble,
  title={Ensemble Learning and 3D Pix2Pix for Comprehensive Brain Tumor Analysis in Multimodal MRI},
  author={Zeineldin et al. (2024)},
  year={2024},
  note={arXiv:2412.11849}
}

• arXiv: 2412.11849