qhjqhj00 histopath-domain-gen-eval
Self-supervised Vision Transformer are Scalable Generative Models for Domain Generalization — Doerrich et al. (2024) (arXiv:2407.02900, 2024)
What this evaluates
Evaluates a model's ability to generalize to out-of-distribution domains (different hospitals or staining protocols) in histopathology image classification. It measures classification accuracy on held-out OOD validation and test splits, alongside the reconstruction quality of self-supervised generative augmentation.
Datasets
- CAMELYON17-WILDS — total 422394; splits: train (302436), val (34904), test (85054)
- Epithelium-Stroma — total 15633; splits: train (-1), val (-1), test (1376)
Metrics
Accuracy (%)(primary) — range: percent- Percentage of correctly classified samples out of the total samples in the validation or test set.
PSNR (dB)— range: other- Peak Signal-to-Noise Ratio measured in decibels to quantify the reconstruction quality of the self-supervised image synthesizer.
Input / output format
Input: 224x224 image patches standardized via bicubic interpolation, with binary class labels (tumor presence or epithelium/stroma).
Output: Binary class prediction for classification; reconstructed or synthetically mixed images for qualitative assessment.
Scoring recipe
def compute_accuracy(predictions, gold_labels):
correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
return (correct / len(gold_labels)) * 100
def compute_psnr(original, reconstructed):
mse = np.mean((original - reconstructed) ** 2)
if mse == 0: return 100
return 10 * np.log10(255**2 / mse)
Common pitfalls
- Using in-distribution data for final evaluation instead of the specified OOD validation/test splits.
- Confusing the generative augmentation phase (training-time mixing of anatomy and characteristics) with the direct classification evaluation phase.
- Failing to standardize all input patches to 224x224 before feeding them to the ViT encoder, which breaks the model's expected input dimensions.
Evidence (verbatim from paper)
We evaluate our method on the class-balanced CAMELYON17-WILDS validation and test sets against the top-performing methods from the WILDS leaderboard, which utilize the same classifier. The results shown in Table 1 reveal our method's superior accuracy on both sets, setting a new state-of-the-art standard.
Citation
@misc{doerrich2024selfsupervised,
title={Self-supervised Vision Transformer are Scalable Generative Models for Domain Generalization},
author={Doerrich et al. (2024)},
year={2024},
note={arXiv:2407.02900}
}
- arXiv: 2407.02900