Rasd Medical Image Benchmark Eval

rasd-medical-image-benchmark-eval

Free Lunch in Medical Image Foundation Model Pre-training via Randomized Synthesis and Disentanglement — Wei et al. (2026) (arXiv:2602.12317, 2026)

What this evaluates

This evaluation probes the transferability and generalization of medical image foundation models pre-trained exclusively on randomized synthetic data. It measures performance across diverse anatomical regions, imaging modalities (CT, MR, X-ray, ultrasound, fundus), and downstream tasks including segmentation, classification, and detection.

Datasets

• TotalSegmentator — total ?; splits: test (-1)
• CHAOS — total ?; splits: test (-1)
• LUNA16 — total ?; splits: test (-1)
• INbreast — total ?; splits: test (-1)
• STARE — total ?; splits: test (-1)
• DDTI — total ?; splits: test (-1)

Metrics

• Dice score (primary) — range: [0, 1]
  • 2 * |A ∩ B| / (|A| + |B|), measuring overlap between predicted and ground truth segmentation masks.
• AUC (primary) — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve, evaluating binary classification performance across all classification thresholds.
• IoU — range: [0, 1]
  • Intersection over Union, calculated as |A ∩ B| / |A ∪ B|, used for segmentation and object detection tasks.

Input / output format

Input: 2D or 3D medical images (CT, MR, X-ray, ultrasound, or fundus) passed through a pre-trained MIFM backbone with task-specific heads.

Output: Per-instance predictions: segmentation masks, classification labels/scores, or detection bounding boxes/masks.

Scoring recipe

def evaluate(predictions, ground_truth, task):
    if task == 'segmentation':
        dice = 2 * np.sum(pred & gt) / (np.sum(pred) + np.sum(gt))
        iou = np.sum(pred & gt) / np.sum(pred | gt)
        return {'dice': dice, 'iou': iou}
    elif task == 'classification':
        auc = roc_auc_score(gt_labels, pred_scores)
        return {'auc': auc}
    elif task == 'detection':
        iou = compute_detection_iou(pred_boxes, gt_boxes)
        return {'iou': iou}

Common pitfalls

• Models are evaluated on real datasets despite being pre-trained solely on synthetic data; performance gaps should not be misinterpreted as overfitting to real data.
• Metrics are reported per dataset/task across 48 datasets; aggregating scores without weighting by dataset size or task difficulty may skew generalization claims.
• IoU is used for both segmentation and detection, which can cause confusion when comparing against literature that reports mAP for detection.

Evidence (verbatim from paper)

For segmentation, their backbones were integrated into Swin UNETR, while for classification, the backbones were paired with a classification head following [wu2024large]. Dice score and AUC were used for the evaluation of segmentation and classification tasks.

Citation

@misc{wei2026rasd,
  title={Free Lunch in Medical Image Foundation Model Pre-training via Randomized Synthesis and Disentanglement},
  author={Wei et al. (2026)},
  year={2026},
  note={arXiv:2602.12317}
}

• arXiv: 2602.12317