Promise2012 Prostate Seg Eval

promise2012-prostate-seg-eval

V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation — Milletari et al. (2016) (arXiv:1606.04797, 2016)

What this evaluates

Evaluates 3D volumetric medical image segmentation capability on prostate MRI scans. It probes the model's ability to accurately delineate organ boundaries under clinical variability and class imbalance using end-to-end fully convolutional networks.

Datasets

• PROMISE2012 — total 80; splits: train (50), test (30)

Metrics

• Dice coefficient (primary) — range: [0, 1]
  • 2 * |prediction ∩ ground_truth| / (|prediction| + |ground_truth|). Measures voxel-wise overlap between the predicted segmentation mask and the manual annotation.
• Hausdorff distance — range: mm
  • Maximum distance between the boundaries of the predicted and ground truth masks. Computed as the largest of the shortest distances from any point on one boundary to the other boundary.
• Score on challenge task — range: other
  • Composite evaluation metric computed by the PROMISE2012 challenge organizers based on segmentation accuracy and robustness.

Input / output format

Input: 3D MRI volumes (prostate) normalized via N4 bias field correction and resampled to a common resolution of 1x1x1.5 mm.

Output: 3D segmentation mask/delineation of the prostate gland.

Scoring recipe

def compute_dice(pred_mask, gt_mask):
    intersection = np.sum(pred_mask * gt_mask)
    return 2.0 * intersection / (np.sum(pred_mask) + np.sum(gt_mask))

def compute_hausdorff(pred_mask, gt_mask):
    pred_boundary = get_boundary(pred_mask)
    gt_boundary = get_boundary(gt_mask)
    dists = cdist(pred_boundary, gt_boundary)
    return max(np.max(np.min(dists, axis=1)), np.max(np.min(dists, axis=0)))

# Aggregate over test set
avg_dice = np.mean([compute_dice(p, g) for p, g in zip(predictions, ground_truths)])
avg_hd = np.mean([compute_hausdorff(p, g) for p, g in zip(predictions, ground_truths)])

Common pitfalls

• Test set ground truth is secret and only provided by challenge organizers, preventing independent verification or local metric calculation.
• Small dataset size (50 train, 30 test) requires heavy data augmentation (random non-linear deformations) to prevent overfitting and ensure clinical generalization.
• Memory constraints force mini-batch size of 2 volumes, which can affect gradient estimation stability and training convergence compared to larger batches.

Evidence (verbatim from paper)

We trained our method on $50$ MRI volumes, and the relative manual ground truth annotation, obtained from the ”PROMISE2012” challenge dataset. ... We evaluated the approach performance in terms of Dice coefficient, Hausdorff distance of the predicted delineation to the ground truth annotation and in terms of score obtained on the challenge data as computed by the organisers of ”PROMISE 2012”.

Citation

@misc{milletari2016vnet,
  title={V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation},
  author={Milletari et al. (2016)},
  year={2016},
  note={arXiv:1606.04797}
}

• arXiv: 1606.04797