Autopet2022 Eval

autopet2022-eval

Exploring Vanilla U-Net for Lesion Segmentation from Whole-body FDG-PET/CT Scans — Ye et al. (2022) (arXiv:2210.07490, 2022)

What this evaluates

Evaluates the capability of 3D medical image segmentation models to accurately delineate lesions in whole-body FDG-PET/CT scans. It probes the model's ability to handle high-resolution volumetric data and distinguish pathological regions from healthy tissue.

Datasets

• autoPET 2022 — total ?; splits: preliminary test (-1), final test (-1), validation (103)

Metrics

• DSC (primary) — range: [0, 1]
  • Dice Similarity Coefficient: 2 * |A ∩ B| / (|A| + |B|), where A and B are the predicted and ground truth lesion masks.
• FPV — range: other
  • False Positive Volume: number of voxels predicted as lesion but belonging to healthy tissue.
• FNV — range: other
  • False Negative Volume: number of voxels belonging to the ground truth lesion but missed by the prediction.

Input / output format

Input: 3D volumetric FDG-PET/CT image patches (typically cropped to 192×192×192 during training) with original or resampled spacing.

Output: 3D binary segmentation mask indicating lesion voxels.

Scoring recipe

def compute_metrics(pred_mask, gt_mask):
    intersection = np.sum(pred_mask & gt_mask)
    union = np.sum(pred_mask) + np.sum(gt_mask)
    dsc = 2 * intersection / union if union > 0 else 0.0
    fpv = np.sum(pred_mask & ~gt_mask)
    fnv = np.sum(~pred_mask & gt_mask)
    return dsc, fpv, fnv

Common pitfalls

• DSC is insensitive to spatial localization errors when lesion volumes are small; FPV and FNV must be reported alongside it.
• Inference step size and crop dimensions heavily influence FPV/FNV trade-offs without significantly altering DSC scores.
• The challenge uses a hidden final test set; preliminary leaderboard rankings do not guarantee final placement due to different evaluation splits.

Evidence (verbatim from paper)

The vanilla U-Net outperforms other architectures for DSC and FPV on the preliminary test set.

Citation

@misc{ye2022exploring,
  title={Exploring Vanilla U-Net for Lesion Segmentation from Whole-body FDG-PET/CT Scans},
  author={Ye et al. (2022)},
  year={2022},
  note={arXiv:2210.07490}
}

• arXiv: 2210.07490