Inbreast Mammogram Classification Eval

inbreast-mammogram-classification-eval

Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification — Zhu et al. (2017) (arXiv:1705.08550, 2017)

What this evaluates

Evaluates the ability of deep multi-instance learning models to classify whole mammograms as benign or malignant without requiring region-of-interest (ROI) annotations. It probes patch-level malignancy prediction and whole-image classification robustness under sparse label conditions.

Datasets

• INbreast — total 410; splits: train (-1), val (-1), test (-1); repo https://github.com/wentaozhu/deep-mil-for-whole-mammogram-classification.git

Metrics

• Accuracy (primary) — range: [0, 1]
  • Fraction of correctly classified mammograms (benign vs. malignant) averaged over 5-fold cross-validation. Reported as mean ± standard deviation.
• AUC — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across classification thresholds. Averaged over 5 folds.

Input / output format

Input: Whole mammogram images resized to 227×227 pixels. Processed through a pretrained AlexNet backbone (fully connected layers removed) to extract 256×6 feature maps, which are then fed into a multi-instance learning (MIL) aggregation module.

Output: Binary classification label (benign/malignant) for the whole mammogram, derived from patch-level malignant probabilities aggregated via max pooling, label assignment, or sparse MIL.

Scoring recipe

folds = split_dataset(INbreast, n_folds=5)
accuracies, aucs = [], []
for train_idx, val_idx, test_idx in folds:
    model = train_mil_model(INbreast[train_idx], INbreast[val_idx])
    preds, probs = predict(model, INbreast[test_idx])
    acc = accuracy_score(INbreast[test_idx].labels, preds)
    auc = roc_auc_score(INbreast[test_idx].labels, probs)
    accuracies.append(acc)
    aucs.append(auc)
final_acc = np.mean(accuracies)
final_auc = np.mean(aucs)

Common pitfalls

• The dataset uses 5-fold cross-validation, not a fixed train/val/test split; results must be averaged across all 5 folds.
• Models are evaluated on whole mammograms without ROI bounding boxes; comparing against methods that use manual ROI annotations is unfair unless explicitly noted.
• Performance metrics (Accuracy and AUC) are reported as mean ± standard deviation over 5 folds, not single-run values.

Evidence (verbatim from paper)

We validate the proposed models on the most frequently used mammographic mass classification dataset, INbreast dataset [14]... The INbreast dataset contains 410 mammograms of which 100 containing malignant masses. ... For fair comparison, we also use 5-fold cross validation to evaluate model performance as [6]. For each testing fold, we use three folds for training, and one fold for validation to tune hyper-parameters. The performance is reported as the average of five testing results obtained from cross-validation. Table 1. Accuracy Comparisons of the proposed deep MILs and related methods on test sets. ... Accu. AUC

Citation

@misc{zhu2017deepmil,
  title={Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification},
  author={Zhu et al. (2017)},
  year={2017},
  note={arXiv:1705.08550}
}

• arXiv: 1705.08550