Mots Segmentation Eval

mots-segmentation-eval

Learning from partially labeled data for multi-organ and tumor segmentation — Xie et al. (2022) (arXiv:2211.06894, 2022)

What this evaluates

Evaluates a model's ability to perform multi-organ and tumor segmentation on partially labeled 3D medical images. It probes the network's capacity to learn from incomplete annotations and generalize across diverse anatomical structures using a unified architecture.

Datasets

• MOTS (Multi-Organ and Tumor Segmentation) — total 1155; splits: train (920), test (235)
• BCV (MICCAI 2015 Multi Atlas Labeling Beyond the Cranial Vault) — total 50; splits: train (30), test (20)
• BraTS (2018 Brain Tumor Segmentation Challenge) — total 351; splits: train (285), test (66)

Metrics

• mean Dice (mDice) (primary) — range: [0, 1]
  • Average of Dice similarity coefficients across all organ and tumor categories. Dice = 2|A∩B|/(|A|+|B|), where A and B are the predicted and ground truth masks.
• mean HD (mHD) — range: mm
  • Average of Hausdorff distances across all organ and tumor categories. HD evaluates boundary quality by computing the maximum distance between predicted and ground truth segmentation boundaries.

Input / output format

Input: 3D medical images (CT or MRI). CT scans are truncated to HU [-325, +325], normalized to [-1, +1], and re-sliced to 1.5×0.8×0.8 mm³. Training inputs are randomly cropped sub-volumes of 64×192×192. Test inputs use a sliding window of 64×256×256.

Output: Voxel-wise segmentation masks for each target organ or tumor category.

Scoring recipe

def compute_metrics(pred_masks, gt_masks, target_organs):
    dice_scores = []
    hd_scores = []
    for organ in target_organs:
        pred = pred_masks[organ]
        gt = gt_masks[organ]
        intersection = np.sum(pred & gt)
        dice = 2 * intersection / (np.sum(pred) + np.sum(gt) + 1e-6)
        dice_scores.append(dice)
        hd = compute_hausdorff_distance(pred, gt)
        hd_scores.append(hd)
    mDice = np.mean(dice_scores)
    mHD = np.mean(hd_scores)
    return mDice, mHD

Common pitfalls

• Partially labeled data means some scans lack annotations for certain organs; the evaluation must handle missing labels during training and only score on available ground truth.
• Hausdorff distance (HD) is highly sensitive to boundary noise and outliers, which can disproportionately skew mHD compared to Dice.
• The sliding window inference strategy requires careful overlap handling and post-processing to avoid boundary artifacts in the final masks.

Evidence (verbatim from paper)

The Dice similarity coefficient (Dice) and Hausdorff distance (HD) are used as performance metrics for this study. Dice measures the overlapping between a segmentation prediction and ground truth, and HD evaluates the quality of segmentation boundaries by computing the maximum distance between the predicted boundaries and ground truth. We assess all segmentation methods based on the mean Dice (mDice)) and mean HD (mHD) across all organ and tumor categories.

Citation

@misc{xie2022learning,
  title={Learning from partially labeled data for multi-organ and tumor segmentation},
  author={Xie et al. (2022)},
  year={2022},
  note={arXiv:2211.06894}
}

• arXiv: 2211.06894