Prior Loss Seg Benchmark

prior-loss-seg-benchmark

Effect of Prior-based Losses on Segmentation Performance: A Benchmark — El Jurdi et al. (2022) (arXiv:2201.02428, 2022)

What this evaluates

Evaluates the effectiveness of various prior-based loss functions (low-level boundary/distance and high-level shape/size constraints) for medical image segmentation across diverse anatomical structures and imaging modalities.

Datasets

• WMH — total ?; splits: test (-1)
• ISLES — total ?; splits: test (-1)
• Atrium — total ?; splits: test (-1)
• Colon — total ?; splits: test (-1)
• Spleen — total ?; splits: test (-1)
• Hippocampus — total ?; splits: test (-1)
• Prostate — total ?; splits: test (-1)
• ACDC — total ?; splits: test (-1)

Metrics

• Dice score (primary) — range: [0, 1]
  • 2 * |A ∩ B| / (|A| + |B|), where A and B are the predicted and ground truth segmentation masks respectively.
• Hausdorff distance — range: other
  • Maximum of the directed distances between the boundaries of the predicted and ground truth masks: max(sup_{a∈A} inf_{b∈B} d(a,b), sup_{b∈B} inf_{a∈A} d(a,b)).
• MAE on connected components — range: other
  • Average absolute difference between the number of connected components (instances) in the ground truth and the predicted segmentation map across all samples.

Input / output format

Input: 3D medical image volumes with corresponding ground truth segmentation masks.

Output: Predicted segmentation map of the same spatial dimensions as the input image.

Scoring recipe

def compute_metrics(pred_mask, gt_mask):
    intersection = np.sum(pred_mask & gt_mask)
    dice = 2.0 * intersection / (np.sum(pred_mask) + np.sum(gt_mask))
    pred_boundary = get_boundary(pred_mask)
    gt_boundary = get_boundary(gt_mask)
    hd = max(max_distance_to_set(pred_boundary, gt_boundary),
             max_distance_to_set(gt_boundary, pred_boundary))
    pred_cc = count_connected_components(pred_mask)
    gt_cc = count_connected_components(gt_mask)
    mae_cc = abs(pred_cc - gt_cc)
    return dice, hd, mae_cc

Common pitfalls

• Hausdorff loss computation is highly computationally expensive during training, significantly increasing training time.
• High-level priors like size loss degrade performance on datasets with large organ size variability or multi-label segmentation tasks.
• clDice optimizes topological skeletons, which can blur boundary details and worsen HD scores despite achieving good Dice accuracy.

Evidence (verbatim from paper)

The segmentation performances are compared via the 2 usual segmentation metrics: the Dice score $^{30}$ (DSC) presented in Table 2, the Hausdorff distance metric $^{31}$ (HD) presented in Table 3. In addition, we have computed the mean absolute error on the number of instances (connected components) presented in Table 4.

Citation

@misc{eljurdi2022effect,
  title={Effect of Prior-based Losses on Segmentation Performance: A Benchmark},
  author={El Jurdi et al. (2022)},
  year={2022},
  note={arXiv:2201.02428}
}

• arXiv: 2201.02428