Brats Oasis1 Uncertainty Eval

brats-oasis1-uncertainty-eval

Enhancing Neuro-Oncology Through Self-Assessing Deep Learning Models for Brain Tumor Unified Model for MRI Segmentation — Zhou (2025) (arXiv:2511.12801, 2025)

What this evaluates

Evaluates a deep learning model's ability to perform multi-region brain MRI segmentation (tumors and healthy structures) while simultaneously predicting per-voxel uncertainty. It probes the model's segmentation accuracy across anatomical regions and its calibration of confidence estimates against actual voxel-wise errors.

Datasets

• BraTS — total ?; splits: test (-1)
• OASIS-1 — total ?; splits: test (-1)

Metrics

• DSC (primary) — range: [0, 1]
  • Dice Similarity Coefficient measuring overlap between predicted and ground truth segmentation masks. Calculated as 2 * |prediction ∩ ground_truth| / (|prediction| + |ground_truth|). Reported per region (e.g., whole tumor, tumor core, enhancing tumor, whole brain).
• RMSD — range: other
  • Root Mean Square Deviation between the model's predicted uncertainty map and the true voxel-wise segmentation error (absolute difference between prediction and ground truth). Lower values indicate better localized uncertainty calibration.
• CORR — range: other
  • Pearson correlation coefficient between the predicted uncertainty map and the true voxel-wise segmentation error. Higher values indicate better rank-order calibration of confidence.

Input / output format

Input: 3D MRI volumes (single modality for unified models, four modalities for cancer-only models).

Output: Multi-label segmentation mask (54 labels including tumor and healthy brain structures) and a continuous per-voxel uncertainty map.

Scoring recipe

def compute_metrics(pred_mask, gt_mask, uncertainty_map):
    # DSC per region
    intersection = np.sum((pred_mask == region_id) & (gt_mask == region_id))
    union = np.sum(pred_mask == region_id) + np.sum(gt_mask == region_id)
    dsc = 2 * intersection / union
    # True error map
    true_error = np.abs(pred_mask - gt_mask)
    # RMSD
    rmsd = np.sqrt(np.mean((uncertainty_map - true_error) ** 2))
    # CORR
    corr = np.corrcoef(uncertainty_map.flatten(), true_error.flatten())[0, 1]
    return dsc, rmsd, corr

Common pitfalls

• DSC is highly sensitive to class imbalance; small structures like the hippocampus or enhancing tumor may show disproportionately low scores despite clinically acceptable performance.
• Uncertainty metrics (RMSD, CORR) require a well-defined 'true error' map; the paper uses voxel-wise absolute difference between prediction and ground truth, which may not align with other uncertainty evaluation protocols that use ensemble variance or Monte Carlo dropout.
• The unified model was trained on only one MRI modality for compatibility with OASIS-1, which limits direct performance comparison with 4-modality baselines like CM1/CM2.

Evidence (verbatim from paper)

Table[1] reports the cancer segmentation DSC for the different brain tumor regions and the uncertainty performance metric for the CM and UM models. ... Metrics demonstrate that the UM1 model excels at uncertainty prediction, as its Pearson correlation coefficient of 0.75 represents state-of-the-art performance compared to other studies (below). The CM1 model achieved a correlation coefficient of 0.634 for uncertainty prediction, which is lower than UM1 but still remarkable considering the model had to segment 54 different region labels. Interestingly, the voxel-wise uncertainty RMSD metrics reveal a different trend: UM1 achieved 0.047 while CM1 achieved 0.032, with CM1 actually performing better. In fact, the RMSD metric reported for CM1 is the lowest among the studies compared in Table[2]. The RMSD value provides a direct voxel-wise evaluation between predicted uncertainty and real error and indicates that our models, particularly CM1, performed better than existing statistics-based methods in localized uncertainty prediction.

Citation

@misc{zhou2025enhancing,
  title={Enhancing Neuro-Oncology Through Self-Assessing Deep Learning Models for Brain Tumor Unified Model for MRI Segmentation},
  author={Zhou (2025)},
  year={2025},
  note={arXiv:2511.12801}
}

• arXiv: 2511.12801