Nuclei Seg Ocda Eval

nuclei-seg-ocda-eval

Learning to Generalize over Subpartitions for Heterogeneity-aware Domain Adaptive Nuclei Segmentation — Fan et al. (2024) (arXiv:2401.09496, 2024)

What this evaluates

Evaluates domain adaptive nuclei instance segmentation under cross-modality and cross-stain settings. It probes the model's ability to generalize to unseen cancer subdomains and different imaging modalities without target annotations.

Datasets

• BBBC039 — total 150; splits: train (100), val (50)
• Kumar — total 30; splits: train (16), test (14)
• CPM17 — total 64; splits: train (32), test (32)
• DataSeg — total 52; splits: train (52)

Metrics

• Panoptic Quality (PQ) (primary) — range: [0, 1]
  • PQ = DQ × SQ, where DQ measures detection accuracy (TP / (TP + 0.5FP + 0.5FN)) and SQ measures segmentation accuracy (average IoU of matched instances). It unifies instance detection and pixel-level segmentation into a single score.
• DICE — range: [0, 1]
  • 2 * |A ∩ B| / (|A| + |B|), measuring pixel-wise overlap between predicted and ground truth masks at the semantic level.
• AJI — range: [0, 1]
  • Adjusted Jaccard Index, an instance-level metric that computes the union of intersections over the union of ground truth masks, penalizing over-segmentation and fragmentation.
• DQ — range: [0, 1]
  • Detection Quality, the ratio of correctly detected instances (IoU > 0.5) to the sum of true positives, false positives, and false negatives.
• SQ — range: [0, 1]
  • Segmentation Quality, the average IoU of matched predicted and ground truth instances.

Input / output format

Input: 256×256 image patches extracted from source domain (fluorescence microscopy or IHC-stained) and target domain (H&E-stained histopathology).

Output: Instance segmentation masks and bounding boxes for each detected nucleus, typically generated via a Mask R-CNN backbone.

Scoring recipe

def compute_pq(pred_masks, gt_masks, iou_thresh=0.5):
    tp, fp, fn = 0, 0, 0
    sq_sum = 0.0
    used_gt = set()
    for i, pred in enumerate(pred_masks):
        best_iou, best_idx = 0, -1
        for j, gt in enumerate(gt_masks):
            if j not in used_gt:
                iou = intersection_over_union(pred, gt)
                if iou > best_iou:
                    best_iou, best_idx = iou, j
        if best_iou >= iou_thresh:
            tp += 1
            sq_sum += best_iou
            used_gt.add(best_idx)
        else:
            fp += 1
    fn = len(gt_masks) - len(used_gt)
    dq = tp / (tp + 0.5 * fp + 0.5 * fn) if (tp + 0.5 * fp + 0.5 * fn) > 0 else 0.0
    sq = sq_sum / tp if tp > 0 else 0.0
    return dq * sq

Common pitfalls

• Unseen cancer subdomains are strictly held out in the test set to evaluate open compound generalization, so standard cross-validation is inappropriate.
• Cross-stain adaptation uses IHC-stained images as source and H&E as target, which differs from the cross-modality fluorescence-to-H&E setup.
• Metrics are reported with standard deviations, implying multiple random seeds or data splits were used, but the exact number of runs is not specified in the text.

Evidence (verbatim from paper)

For the purpose of fair comparison, we adopt three metrics to evaluate the performance of nuclei instance segmentation which are broadly used in previous works. Panoptic quality (PQ) is a unified score which integrates detection quality (DQ) and segmentation (SQ) [70]. The consolidated metric inherits capability to simultaneously measure the accuracy with respect to both detection and segmentation tasks. It is considered as a robust quantification for comprehensive evaluation of instance segmentation result. Additionally, we adopt DICE and AJI [18] to perform supplemental evaluation at semantic and instance level respectively.

Citation

@misc{fan2024learning,
  title={Learning to Generalize over Subpartitions for Heterogeneity-aware Domain Adaptive Nuclei Segmentation},
  author={Fan et al. (2024)},
  year={2024},
  note={arXiv:2401.09496}
}

• arXiv: 2401.09496