Pam50 Subtyping Eval

pam50-subtyping-eval

Scalable and Loosely-Coupled Multimodal Deep Learning for Breast Cancer Subtyping — Amer et al. (2025) (arXiv:2509.03408, 2025)

What this evaluates

Evaluates a multimodal deep learning framework's ability to classify breast cancer into four PAM50 molecular subtypes using whole slide images, copy number variation data, and clinical records. The protocol tests how well late fusion of heterogeneous biomedical modalities handles class imbalance and spatial-graph features for diagnostic subtyping.

Datasets

• TCGA-BRCA — total 977; splits: test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • The proportion of correctly classified patients out of the total evaluated in a fold. Calculated as (True Positives + True Negatives) / Total.
• macro-AUROC — range: [0, 1]
  • The average of the area under the receiver operating characteristic curve computed independently for each of the four PAM50 subtypes, then averaged across classes.

Input / output format

Input: Per-patient multimodal input: whole slide image (WSI) features extracted via Inceptionv3, copy number variation (CNV) vectors, and clinical/EHR tabular data. Each modality is processed by a dedicated encoder before weighted logits late fusion.

Output: Predicted PAM50 subtype class (Luminal A, Luminal B, Basal-like, or Her2-enriched) derived from fused modality logits.

Scoring recipe

def evaluate(predictions, labels):
    # predictions: list of predicted class indices or logits
    # labels: list of true class indices (0-3)
    accuracy = sum(p == l for p, l in zip(predictions, labels)) / len(labels)
    # macro-AUROC: average AUC across 4 classes
    macro_auroc = np.mean([roc_auc_score(labels == c, predictions[:, c]) for c in range(4)])
    return {'accuracy': accuracy, 'macro-AUROC': macro_auroc}

Common pitfalls

• The paper mentions three class imbalance mitigation strategies (oversampling, stratified sampling, loss weighting) but does not explicitly state which one was used for the final reported results.
• 10-fold cross-validation is used, but exact train/test split sizes per fold are not provided, making direct replication of fold boundaries difficult without the original data partitioning code.
• Macro-AUROC is computed across four highly imbalanced classes, which can be sensitive to the minority class (Her2-enriched at 7.8%).

Evidence (verbatim from paper)

Evaluation uses 10-fold cross-validation, reporting accuracy and macro-AUROC (average AUC across classes).

Citation

@misc{amer2025subtyping,
  title={Scalable and Loosely-Coupled Multimodal Deep Learning for Breast Cancer Subtyping},
  author={Amer et al. (2025)},
  year={2025},
  note={arXiv:2509.03408}
}

• arXiv: 2509.03408