Dosrecmc Mammography Eval

dosrecmc-mammography-eval

DoSReMC: Domain Shift Resilient Mammography Classification using Batch Normalization Adaptation — Akyüz et al. (2025) (arXiv:2508.15452, 2025)

What this evaluates

Evaluates cross-domain generalization of mammography classification models under domain shift, specifically testing resilience to variations in pixel intensity distributions across different imaging devices and datasets.

Datasets

• NYU — total ?; splits: train (-1)
• HCTP — total ?; splits: test (-1)
• VinDr — total ?; splits: test (-1)
• CSAW — total ?; splits: test (-1)

Metrics

• PR-AUC (primary) — range: [0, 1]
  • Area under the Precision-Recall curve computed specifically on the malignant class predictions. It emphasizes the model's ability to minimize false positives and false negatives for malignant cases.
• ROC-AUC — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve computed on the malignant class predictions. It measures the trade-off between the true positive rate and false positive rate across classification thresholds.

Input / output format

Input: Mammography images (CC and MLO views) processed into 256×256 patches. Training batches are composed of benign, malignant, and negative samples in a 1:1:2 ratio.

Output: Classification probabilities for benign, malignant, and negative classes. Final prediction values are obtained by averaging the outputs for CC and MLO images of the same breast.

Scoring recipe

def compute_pr_auc(predictions_cc, predictions_mlo, labels):
    # Average predictions for paired CC and MLO views
    avg_pred = [(c + m) / 2 for c, m in zip(predictions_cc, predictions_mlo)]
    # Extract malignant class scores and true labels
    y_true = [1 if label == 'malignant' else 0 for label in labels]
    y_score = [p[1] for p in avg_pred] # assuming index 1 is malignant
    # Compute PR-AUC
    precision, recall, _ = precision_recall_curve(y_true, y_score)
    return auc(recall, precision)

Common pitfalls

• Failing to average the CC and MLO view predictions for the same breast before computing metrics.
• Evaluating on all classes equally instead of focusing specifically on the malignant class for PR-AUC.
• Using full model fine-tuning instead of the specified BN and FC layer adaptation protocol.

Evidence (verbatim from paper)

In order to evaluate the model’s performance, we use the Receiver Operator Characteristic - Area Under Curve (ROC-AUC) and PR-AUC. While evaluating the model, we average the predictions corresponding to CC and MLO images of a breast in the same study, and obtain final prediction values. Our primary goal is to achieve low false-positive and false-negative rates for malignant cases, and we focus on the model’s outputs for malignant cases during the analysis phase.

Citation

@misc{akyuz2025dosrecmc,
  title={DoSReMC: Domain Shift Resilient Mammography Classification using Batch Normalization Adaptation},
  author={Akyüz et al. (2025)},
  year={2025},
  note={arXiv:2508.15452}
}

• arXiv: 2508.15452