Pancreas Segmentation Pruning Eval

pancreas-segmentation-pruning-eval

Data-Centric Diet: Effective Multi-center Dataset Pruning for Medical Image Segmentation — He et al. (2023) (arXiv:2308.01189, 2023)

What this evaluates

This protocol evaluates the effectiveness of data pruning strategies for 3D medical image segmentation. It measures how well a pruned training subset preserves model performance on pancreas segmentation tasks compared to using the full dataset or random subsets, specifically testing whether early-training dynamics can guide efficient sample selection without accuracy loss.

Datasets

• MSD-Pancreas — total 100; splits: train (80), test (20)
• WORD — total 100; splits: train (80), test (20); repo https://github.com/HiLab-git/WORD
• NIH-Pancreas — total 82; splits: train (60), test (22)

Metrics

• DSC score (primary) — range: [0, 1]
  • Dice Similarity Coefficient, calculated as 2 * |prediction ∩ ground_truth| / (|prediction| + |ground_truth|). Measures voxel-wise overlap between the predicted segmentation mask and the manual annotation.

Input / output format

Input: 3D CT scan patches of size 64×64×64 with corresponding binary segmentation masks for the pancreas.

Output: Voxel-wise probability map or binary segmentation mask indicating the predicted pancreas region.

Scoring recipe

def compute_dsc_score(pred_mask, gold_mask):
    pred = (pred_mask > 0.5).astype(np.int32)
    gold = (gold_mask > 0).astype(np.int32)
    intersection = np.sum(pred * gold)
    union = np.sum(pred) + np.sum(gold)
    if union == 0:
        return 0.0
    return 2.0 * intersection / union

Common pitfalls

• Applying standard gradient-based importance metrics (e.g., VOG, EL2N) directly to dense medical labels fails because background voxels dominate the gradient signal.
• Assuming that retaining only the hardest-to-learn samples improves pruning; the protocol demonstrates that ambiguous samples yield better performance on small datasets.
• Computing pruning scores after full model convergence; sample difficulty rankings change dynamically during training, so early-epoch computation is recommended for efficiency.

Evidence (verbatim from paper)

Table 1: Comparison of DSC score for V-Net models trained on different regions selected by DAD score. The results show that models trained on those ambiguous samples performed better.

Citation

@misc{he2023datacentricdiet,
  title={Data-Centric Diet: Effective Multi-center Dataset Pruning for Medical Image Segmentation},
  author={He et al. (2023)},
  year={2023},
  note={arXiv:2308.01189}
}

• arXiv: 2308.01189