Radioactive Eval

radioactive-eval

RadioActive: 3D Radiological Interactive Segmentation Benchmark — Ulrich et al. (2024) (arXiv:2411.07885, 2024)

What this evaluates

Evaluates interactive 3D medical image segmentation by measuring how well models segment target structures under varying human-in-the-loop prompting strategies (points, boxes, scribbles) and iterative refinement protocols. It specifically probes the trade-off between interaction effort and segmentation accuracy across 2D and 3D architectures.

Datasets

• RadioActive — total ?; splits: (unstated); repo https://github.com/MIC-DKFZ/radioactive

Metrics

• Dice (primary) — range: [0, 1]
  • Computes the overlap between the predicted segmentation mask and the ground truth mask, calculated as 2 * |A ∩ B| / (|A| + |B|). Reported as mean or average across cases/slices.

Input / output format

Input: 3D radiological volumes with interactive prompts provided as points, 2D/3D bounding boxes, or scribbles. Prompts are applied either slice-by-slice, per-volume, or via interpolation/propagation between initial prompts.

Output: 3D segmentation mask corresponding to the prompted target structure.

Scoring recipe

def dice_score(pred, gt):
    intersection = np.logical_and(pred, gt).sum()
    union = pred.sum() + gt.sum()
    return 2.0 * intersection / union if union > 0 else 0.0
mean_dice = np.mean([dice_score(p, g) for p, g in zip(predictions, ground_truths)])

Common pitfalls

• Slice-by-slice prompting for 2D models is unrealistic and significantly inflates performance compared to human-in-the-loop scenarios.
• Propagation-based prompting can terminate early or cause severe oversegmentation if an initial prompt fails, unlike interpolation methods.
• Forgetting to re-provide initial prompts during iterative refinement steps leads to performance degradation.

Evidence (verbatim from paper)

Figure 5 shows that models employing box prompts achieved significantly higher average Dice scores, with SAM2 demonstrating the strongest performance across all models. Conversely, point-based prompts performed poorly, particularly for small target regions, such as small MS lesions in dataset D1 (see [Tab. 2] in the appendix).

Citation

@misc{ulrich2024radioactive,
  title={RadioActive: 3D Radiological Interactive Segmentation Benchmark},
  author={Ulrich et al. (2024)},
  year={2024},
  note={arXiv:2411.07885}
}

• arXiv: 2411.07885