Bcs Dbt Classification Eval

bcs-dbt-classification-eval

SIFT-DBT: Self-supervised Initialization and Fine-Tuning for Imbalanced Digital Breast Tomosynthesis Image Classification — Du et al. (2024) (arXiv:2403.13148, 2024)

What this evaluates

Evaluates the ability of self-supervised contrastive pre-training and multi-patch fine-tuning to classify imbalanced digital breast tomosynthesis (DBT) slices and volumes as normal or abnormal. It probes the model's robustness to extreme class imbalance and its capacity to preserve spatial resolution through patch-level processing.

Datasets

• BCS-DBT — total 18432; splits: train (12819), val (1838), test (3775)

Metrics

• AUC (primary) — range: percent
  • Area under the receiver operating characteristic (ROC) curve, measuring the model's overall discriminative ability across all classification thresholds.
• NPV — range: percent
  • Negative Predictive Value: proportion of predicted normal cases that are truly normal.
• Recall — range: percent
  • True positive rate: proportion of actual abnormal cases correctly identified.
• Specificity@87 — range: percent
  • Specificity measured when sensitivity is fixed at 87%.
• Specificity@80 — range: percent
  • Specificity measured when sensitivity is fixed at 80%.

Input / output format

Input: Single DBT slice or a set of N (1 or 20) 448x448 patches extracted from a slice. Slices are resized to 1024px on the short side with background removed via Otsu thresholding.

Output: Binary classification probability/score for each slice or volume (normal vs. abnormal). Volume-level scores are aggregated from slice-level predictions.

Scoring recipe

def compute_metrics(preds, y_true):
    fpr, tpr, _ = roc_curve(y_true, preds)
    auc = auc(fpr, tpr) * 100
    # NPV, Recall, Specificity at fixed sensitivity
    # Use sklearn.metrics to compute at optimal or fixed thresholds
    # Specificity@Sens: find threshold where tpr >= Sens, then compute specificity at that threshold
    return auc, npv, recall, spec_87, spec_80

Common pitfalls

• Accuracy is explicitly excluded due to extreme class imbalance; reporting it would be misleading as a model could achieve high accuracy by predicting all inputs as normal.
• Volume-level evaluation aggregates slice-level predictions rather than training a separate volume-level classifier end-to-end.
• The original 'Actionable' class was removed, merging benign and cancer cases into a single 'abnormal' category.

Evidence (verbatim from paper)

We assess the same performance metrics for both slice- and volume-level classification. We do not report accuracy since the extreme imbalance in data distribution allows a model to produce a high accuracy by simply predicting all inputs to normal. Instead, we report the AUC, Negative Predictive Value (NPV), and Recall for each class, along with specificity at common sensitivity levels of 87% and 80% [15]. All numbers are reported in percentages (%).

Citation

@misc{du2024siftdbt,
  title={SIFT-DBT: Self-supervised Initialization and Fine-Tuning for Imbalanced Digital Breast Tomosynthesis Image Classification},
  author={Du et al. (2024)},
  year={2024},
  note={arXiv:2403.13148}
}

• arXiv: 2403.13148