Vikl Mammography Eval

vikl-mammography-eval

ViKL: A Mammography Interpretation Framework via Multimodal Aggregation of Visual-knowledge-linguistic Features — Wei et al. (2024) (arXiv:2409.15744, 2024)

What this evaluates

Evaluates a model's ability to extract robust visual features from single-channel mammography images for binary classification of malignant versus benign breast lumps. It tests cross-dataset generalization and the effectiveness of multimodal contrastive pretraining on pathological classification tasks.

Datasets

• MVKL — total 2764; splits: train (1946), val (276), test (542)
• CBIS-DDSM — total 3568; splits: train (2577), val (287), test (704)
• INbreast — total 410; splits: train (287), val (41), test (82)

Metrics

• AUC (primary) — range: [0, 1]
  • Area under the receiver operating characteristic curve. It represents the probability that a randomly chosen positive instance (malignant) will be ranked higher than a randomly chosen negative instance (benign) by the classifier.

Input / output format

Input: Single-channel mammography images.

Output: Binary classification prediction (malignant or benign).

Scoring recipe

def compute_auc(y_true, y_scores):
    # y_true: binary ground truth (0=benign, 1=malignant)
    # y_scores: predicted probability of malignancy
    fpr, tpr, _ = roc_curve(y_true, y_scores)
    auc_value = auc(fpr, tpr)
    return auc_value

Common pitfalls

• Ensure strict patient-level splitting: images from the same patient must not appear in both train and test sets to prevent data leakage.
• AUC is threshold-independent; do not report accuracy or F1 as the primary metric, as the paper explicitly avoids thresholding complexities.
• Cross-dataset evaluation faces significant domain shifts due to variations in collection years and imaging equipment, which can cause performance drops compared to in-dataset results.

Evidence (verbatim from paper)

To ensure consistent and clear comparisons between models, we exclusively use the area under the receiver operating characteristic (AUC). This metric provides a straightforward measure of performance without the complexities introduced by thresholding.

Citation

@misc{wei2024vikl,
  title={ViKL: A Mammography Interpretation Framework via Multimodal Aggregation of Visual-knowledge-linguistic Features},
  author={Wei et al. (2024)},
  year={2024},
  note={arXiv:2409.15744}
}

• arXiv: 2409.15744