# Brats Challenge Eval

> brats-challenge-eval

- Skill: `qhjqhj00/brats-challenge-eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add qhjqhj00/brats-challenge-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/brats-challenge-eval/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/qhjqhj00/brats-challenge-eval

---


# brats-challenge-eval

> Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge — Spyridon Bakas et al. (arXiv:1811.02629, 2018)

## What this evaluates

Evaluates machine learning models on three clinical neuro-oncology tasks using multi-modal MRI data: multi-compartment brain tumor segmentation, tumor progression assessment, and overall patient survival prediction.

## Datasets

- **BraTS Challenge** — total ?; splits: test (-1)

## Metrics

- `Dice score` **(primary)** — range: [0, 1]
  - Computes the overlap between predicted and ground-truth segmentation masks. Formula: 2 * |A ∩ B| / (|A| + |B|). Evaluated separately for Whole Tumor (WT), Tumor Core (TC), and Active Tumor (AT).
- `95% Hausdorff distance` — range: mm
  - Measures the maximum boundary discrepancy between predictions and ground truth, computed as the 95th percentile of pairwise distances between surface voxels to reduce outlier sensitivity.
- `Accuracy` — range: [0, 1]
  - Proportion of correctly classified patient survival outcomes among three classes.

## Input / output format

**Input**: 3D MRI volumes per patient.

**Output**: Per instance: (1) Three binary segmentation masks (WT, TC, AT) or (2) One of three survival class labels.

## Scoring recipe

```python
def dice_score(pred_mask, gold_mask):
    intersection = np.sum(pred_mask * gold_mask)
    return 2.0 * intersection / (np.sum(pred_mask) + np.sum(gold_mask))

def accuracy(pred_labels, gold_labels):
    return np.mean(pred_labels == gold_labels)

# For each tumor compartment (WT, TC, AT), compute Dice score.
# For survival task, compute Accuracy across all patients.
```

## Common pitfalls

- Dice score is highly sensitive to small tumor volumes (e.g., AT), causing average Dice to be skewed by outliers compared to median Dice.
- For the 3-class survival task, a random baseline accuracy is 0.33, not 0.0 or 0.5, which must be accounted for when interpreting performance.
- Median vs. average Dice can diverge significantly due to skewness in the distribution of results across teams.

## Evidence (verbatim from paper)

> Specifically, when evaluating WT, TC, and AT segmentation, with respect to Dice score and Hausdorff distance... The 95% Hausdorff distance metric is used to characterize the levels of robustness of the automated results... Overall, the top-5 approaches obtained an accuracy around 0.6, while the rest of teams obtained an accuracy in the range of [0.15-0.55]. We should clarify that the random choice should be considered the 0.33 since this is a 3-class classification.

## Citation

```bibtex
@misc{bakas2018brats,
  title={Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge},
  author={Spyridon Bakas et al.},
  year={2018},
  note={arXiv:1811.02629}
}
```

- arXiv: 1811.02629