Mammography Mass Detection Eval

mammography-mass-detection-eval

Domain generalization in deep learning-based mass detection in mammography: A large-scale multi-center study — Garrucho et al. (2022) (arXiv:2201.11620, 2022)

What this evaluates

Evaluates the ability of deep learning models to detect breast masses in digital mammography images across multiple clinical domains with varying scanner manufacturers and imaging protocols. It specifically probes domain generalization capabilities by measuring detection robustness on unseen data distributions.

Datasets

• OPTIMAM Hologic — total ?; splits: train (-1)
• OPTIMAM Siemens — total ?; splits: test (-1)
• OPTIMAM GE — total ?; splits: test (-1)
• OPTIMAM Philips — total ?; splits: test (-1)
• INbreast — total ?; splits: test (-1)
• BCDR — total ?; splits: test (-1)

Metrics

• TPR at 0.75 FPPI (primary) — range: [0, 1]
  • True Positive Rate calculated at a fixed threshold of 0.75 False Positives Per Image. TPR = TP / (TP + FN).
• AUC (FROC) — range: [0, 1]
  • Area Under the Free-response Receiver Operating Characteristic curve, plotting TPR against FPPI across varying detection thresholds.

Input / output format

Input: Digital mammography images (2D X-ray scans).

Output: Bounding boxes (coordinates) indicating the location and extent of detected breast masses.

Scoring recipe

def evaluate(predictions, gold_boxes, n_images):
    tp = count_true_positives(predictions, gold_boxes)
    fp = count_false_positives(predictions, gold_boxes)
    tpr = tp / len(gold_boxes)
    fppi = fp / n_images
    auc = compute_auc(tpr_values, fppi_values)
    return tpr, fppi, auc

Common pitfalls

• FPPI (false positives per image) normalizes by the number of images, not patches or regions, which differs from standard FPI metrics.
• Class imbalance heavily skews overall TPR; benign masses are rare in source domains (e.g., 9% in OPTIMAM Hologic) but dominate in others (e.g., 55% in BCDR), requiring per-class analysis.
• Domain shift is primarily driven by scanner manufacturer and acquisition protocol rather than pure covariate shift, making cross-domain generalization the core challenge.

Evidence (verbatim from paper)

Table 4 shows, for each domain, the performance of the methods in terms of TPR at 0.75 FPPI, 95% confidence intervals, and AUC. Fig. 4 contains the FROC curves for the six different domains.

Citation

@misc{garrucho2022mammographydg,
  title={Domain generalization in deep learning-based mass detection in mammography: A large-scale multi-center study},
  author={Garrucho et al. (2022)},
  year={2022},
  note={arXiv:2201.11620}
}

• arXiv: 2201.11620