Ovarian Cancer Subtype Eval

ovarian-cancer-subtype-eval

A Comprehensive Evaluation of Histopathology Foundation Models for Ovarian Cancer Subtype Classification — Breen et al. (2024) (arXiv:2405.09990, 2024)

What this evaluates

Evaluates histopathology foundation models and ImageNet-pretrained encoders on classifying ovarian cancer subtypes from whole slide images. It probes the ability of vision models to extract diagnostically relevant features from medical histology slides for multi-class classification.

Datasets

• Ovarian Cancer WSI Dataset — total 1864; splits: cross-validation (-1), hold-out (-1), external_transcanadian (-1), external_ocean (-1); repo https://github.com/scjjb/Ovarian_Features

Metrics

• balanced accuracy (primary) — range: percent
  • The unweighted average of recall across all classes, calculated as the mean of true positive rates per class to handle class imbalance.
• AUROC — range: [0, 1]
  • Area under the receiver operating characteristic curve, measuring the model's ability to discriminate between classes across all classification thresholds.
• F1 Score — range: [0, 1]
  • The harmonic mean of precision and recall, calculated per class and then macro-averaged across all classes.

Input / output format

Input: Whole slide histopathology images (WSIs) from ovarian cancer tissue samples.

Output: Predicted ovarian cancer subtype label (multi-class classification).

Scoring recipe

def compute_metrics(y_true, y_pred, y_prob, num_classes):
    from sklearn.metrics import balanced_accuracy_score, roc_auc_score, f1_score
    bal_acc = balanced_accuracy_score(y_true, y_pred)
    auroc = roc_auc_score(y_true, y_prob, multi_class='ovr')
    f1 = f1_score(y_true, y_pred, average='macro')
    return bal_acc, auroc, f1

Common pitfalls

• Performance is averaged across four distinct validation settings (CV, hold-out, Transcanadian, OCEAN), which may obscure dataset-specific variance or overfitting to specific cohorts.
• RN18-Histo is the only foundation model that underperforms ImageNet-pretrained baselines, contrary to the general trend of histopathology models.
• 95% confidence intervals are derived from 10,000 bootstrap iterations rather than standard cross-validation variance, requiring careful interpretation of statistical significance.

Evidence (verbatim from paper)

The H-optimus-0 model achieved the greatest averaged performance across all validations (Table 3), with 83.0% average balanced accuracy, 0.965 average AUROC, and 0.822 average F1 score.

Citation

@misc{breen2024ovarian,
  title={A Comprehensive Evaluation of Histopathology Foundation Models for Ovarian Cancer Subtype Classification},
  author={Breen et al. (2024)},
  year={2024},
  note={arXiv:2405.09990}
}

• arXiv: 2405.09990