Tcga Histopathology Eval

tcga-histopathology-eval

Domain-specific optimization and diverse evaluation of self-supervised models for histopathology — Lai et al. (2023) (arXiv:2310.13259, 2023)

What this evaluates

Evaluates the representation quality and generalization of self-supervised histopathology models across diverse patch-level diagnostic tasks and weakly supervised slide-level tasks using linear probing and fine-tuning on TCGA whole slide images.

Datasets

• TCGA Histopathology — total ?; splits: test (-1), tune (-1)

Metrics

• average AUC (primary) — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve, averaged across all linear probe tasks. Reported with 95% confidence intervals.

Input / output format

Input: Image patches extracted from whole slide images (WSIs) at varying magnifications. For weakly supervised slide-level tasks, 1000 patches per slide are averaged to form a single slide embedding.

Output: Linear probe predictions (probabilities or logits) for each diagnostic task (e.g., cancer subtype, biomarker status, mitotic figure presence).

Scoring recipe

def compute_average_auc(predictions, labels):
    aucs = []
    for pred, lab in zip(predictions, labels):
        fpr, tpr, _ = roc_curve(lab, pred)
        aucs.append(auc(fpr, tpr))
    return np.mean(aucs)
# predictions: list of arrays, each shape (n_samples,)
# labels: list of arrays, each shape (n_samples,)

Common pitfalls

• Center embeddings significantly outperform class token embeddings for mitotic figure detection, but this is task-specific and not always reflected in overall linear probe averages.
• Ablation variations that improve performance on the tune set may not generalize to held-out test tasks, requiring careful validation on independent diagnostic tasks.

Evidence (verbatim from paper)

Baseline models as well as SimCLR-best and MSN-best (combining individual optimizations) were evaluated on the test sets for all linear probe tasks. See Table 2 for linear probe metric performance for all models. For the optimized models, average AUC was 93.20% [95% CI 92.71% - 93.72%] for SimCLR-best and 93.43% [95% CI 92.88% - 93.84%] for MSN-best.

Citation

@misc{lai2023domain,
  title={Domain-specific optimization and diverse evaluation of self-supervised models for histopathology},
  author={Lai et al. (2023)},
  year={2023},
  note={arXiv:2310.13259}
}

• arXiv: 2310.13259