Optimam Mammography Eval

optimam-mammography-eval

Deep is a Luxury We Don't Have — Ahmed Taha et al. (arXiv:2208.06066, 2022)

What this evaluates

Binary classification of high-resolution mammography images to detect malignant breast tissue. It probes the model's ability to distinguish between malignant and non-malignant cases using both localized patches and full-resolution inputs.

Datasets

• OPTIMAM — total 148448; splits: train (58376), val (7266), test (7358)

Metrics

• AUC (primary) — range: [0, 1]
  • Area under the receiver operating characteristic curve. Computed as the integral of the true positive rate versus false positive rate across all classification thresholds. 95% confidence intervals are estimated via percentile bootstrapping with 10,000 iterations.

Input / output format

Input: High-resolution mammography images (3328x2560) or cropped patches (512x512). Positive patches are centered on malignant bounding boxes; negative patches are randomly sampled from the breast area in non-malignant images.

Output: Binary classification label: malignant (positive) or non-malignant (negative).

Scoring recipe

def compute_auc(predictions, labels):
    fpr, tpr, _ = roc_curve(labels, predictions)
    auc_value = auc(fpr, tpr)
    boot_aucs = []
    for _ in range(10000):
        idx = np.random.choice(len(labels), size=len(labels), replace=True)
        boot_aucs.append(auc(fpr[idx], tpr[idx]))
    ci_low, ci_high = np.percentile(boot_aucs, [2.5, 97.5])
    return auc_value, (ci_low, ci_high)

Common pitfalls

• Patient-level splitting is required to prevent data leakage, as multiple images can come from the same patient.
• Patch-based training uses positively centered patches but randomly sampled negative patches, which may create an artificial distribution shift compared to full-image inference.
• Performance varies significantly between patch-level and full-image evaluation modes; results should not be conflated.

Evidence (verbatim from paper)

Performance is reported using AUC and their 95% confidence intervals (CI) are in square brackets. We evaluate both the small patch and the full image models. The CI is constructed via percentile bootstrapping with 10,000 bootstrap iterations.

Citation

@misc{taha2022deepisluxury,
  title={Deep is a Luxury We Don't Have},
  author={Ahmed Taha et al.},
  year={2022},
  note={arXiv:2208.06066}
}

• arXiv: 2208.06066