Breast Lesion Classification Eval

breast-lesion-classification-eval

Automatic Breast Lesion Classification by Joint Neural Analysis of Mammography and Ultrasound — Habib et al. (2020) (arXiv:2009.11009, 2020)

What this evaluates

Evaluates a model's ability to classify breast lesions as benign or malignant using paired mammography and ultrasound images. It probes multimodal fusion capabilities by comparing single-modality performance against a simple average of combined modality predictions.

Datasets

• Breast Lesion Dataset (153 pairs) — total 153; splits: train (119), test (1), val (33)

Metrics

• AUC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic (ROC) curve, calculated from all test scores across the leave-one-out folds. Ranges from 0 to 1, where 1 indicates perfect discrimination between benign and malignant lesions.

Input / output format

Input: Paired mammography and ultrasound images of a single breast lesion.

Output: A malignancy probability score (0–1) per modality, plus a combined score (average of the two modality scores).

Scoring recipe

scores = []
labels = []
for lesion in test_set:
    pred_mammo = model.predict(lesion.mammo)
    pred_us = model.predict(lesion.us)
    combined = (pred_mammo + pred_us) / 2
    scores.append(combined)
    labels.append(lesion.is_malignant)
auc = roc_auc_score(labels, scores)

Common pitfalls

• The dataset is small (153 pairs), so results are highly sensitive to the specific 120/33 split and hyperparameter tuning on the validation set.
• The combined score is explicitly defined as the arithmetic average of the two modality scores, not a learned fusion weight, which may limit performance compared to end-to-end fusion.
• AUC is computed over all test lesions across LOO folds rather than averaging per-fold AUCs, which can inflate variance if lesion distribution isn't perfectly balanced across folds.

Evidence (verbatim from paper)

Given 153 mammography-ultrasound lesion pairs, we randomly selected 120 fixed pairs for the leave-one-out experiments, benign and malignant being equally distributed. The remaining 33 lesions were held out as validation set for hyper-parameter tuning. ... Finally, the test lesion obtained three scores, one for each modality and one combined, representing the average malignancy probabilities of all its appearances in the dataset. Results were evaluated by means of AUC (area under the ROC curve), calculated from all test scores.

Citation

@misc{habib2020automatic,
  title={Automatic Breast Lesion Classification by Joint Neural Analysis of Mammography and Ultrasound},
  author={Habib et al. (2020)},
  year={2020},
  note={arXiv:2009.11009}
}

• arXiv: 2009.11009