Few Shot Histopathology Eval

few-shot-histopathology-eval

Few-Shot Histopathology Image Classification: Evaluating State-of-the-Art Methods and Unveiling Performance Insights — Sekhar et al. (2024) (arXiv:2408.13816, 2024)

What this evaluates

Evaluates the few-shot classification capability of deep learning models on histopathology images. It probes how well models generalize from extremely limited labeled examples (1, 5, or 10 per class) across disjoint medical imaging domains with varying resolutions and class distributions.

Datasets

• Komura & Ishikawa (2021) — total 1608060; splits: train (1608060)
• CRC-TP — total 280000; splits: test (-1)
• NCT — total 100000; splits: test (-1)
• LC25000 — total 25000; splits: test (-1)

Metrics

• Accuracy(%) (primary) — range: percent
  • Standard classification accuracy calculated as the number of correctly predicted query images divided by the total number of query images across all test episodes, multiplied by 100.

Input / output format

Input: Histopathology image patches (150x150, 224x224, or 768x768 pixels) organized into few-shot episodes containing K support images per class (1, 5, or 10) across 5 classes, plus 15 query images per class.

Output: Predicted class label for each query image in the episode.

Scoring recipe

correct = 0
total = 0
for episode in test_episodes:  # 5000 episodes per dataset
    support = episode.support_images  # K images per class, 5 classes
    queries = episode.query_images    # 15 images per class
    model = train_or_finetune(support)
    for q in queries:
        if model.predict(q) == q.label:
            correct += 1
        total += 1
accuracy = (correct / total) * 100

Common pitfalls

• Training and testing datasets are strictly disjoint (Komura for training, CRC-TP/NCT/LC25000 for testing), so reported Accuracy(%) reflects domain-shift robustness rather than in-distribution generalization.
• Episodic training methods (MAML, ProtoNet) are directly compared to standard fine-tuning methods (SimpleShot, LaplacianShot) without normalizing for the massive 1.6M image training set, which heavily biases standard training procedures.
• Accuracy saturates after 10 shots across all methods; extrapolating performance to higher shot counts is not supported by the evaluation protocol.

Evidence (verbatim from paper)

Table 1: Accuracy(%) on three different datasets; CRC-TP, NCT and LC25000. All the trained models follow meta-testing. In this procedure, 5000 episodes are randomly sampled from CRC-TP, NCT and LC25000 dataset. In all K-shot testing scenarios, the number of query images are fixed as 15 in each episode. The results on 1-shot, 5-shot, 10-shot on different datasets are reported in the Table 1.

Citation

@misc{sekhar2024fewshot,
  title={Few-Shot Histopathology Image Classification: Evaluating State-of-the-Art Methods and Unveiling Performance Insights},
  author={Sekhar et al. (2024)},
  year={2024},
  note={arXiv:2408.13816}
}

• arXiv: 2408.13816