Myriadal Eval

myriadal-eval

MyriadAL: Active Few Shot Learning for Histopathology — Schiavone et al. (2023) (arXiv:2310.16161, 2023)

What this evaluates

Evaluates active few-shot learning frameworks for histopathology image classification under extremely tight annotation budgets (1, 5, and 10 labeled samples). It probes how well uncertainty-based diversity sampling and self-supervised contrastive pretraining can reduce sample redundancy and improve classification performance compared to standard few-shot and active learning baselines.

Datasets

• NCT-CRC-HE-100K — total 100000; splits: train (-1), test (-1)
• BreaKHis — total 9109; splits: train (-1), test (-1)

Metrics

• Accuracy (primary) — range: percent
  • Proportion of correctly classified test images out of the total test set size.
• Macro F1 Score — range: percent
  • Unweighted mean of the F1 scores computed independently for each class, where F1 is the harmonic mean of precision and recall per class.

Input / output format

Input: RGB histopathology image patches (224×224 for NCT, 700×460 for BreaKHis) with ground-truth class labels (9 classes for NCT, 8 classes for BreaKHis).

Output: Predicted class label for each test image.

Scoring recipe

correct = sum(p == g for p, g in zip(predictions, gold))
accuracy = correct / len(gold)
f1_scores = []
for cls in classes:
    tp = sum(p == cls and g == cls for p, g in zip(predictions, gold))
    fp = sum(p == cls and g != cls for p, g in zip(predictions, gold))
    fn = sum(p != cls and g == cls for p, g in zip(predictions, gold))
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
    f1_scores.append(f1)
macro_f1 = sum(f1_scores) / len(classes)

Common pitfalls

• Active learning baselines require an initial seed of K randomly chosen labeled samples; failing to fix this seed across runs breaks comparability.
• All results are reported as mean ± standard deviation over 3 random seeds; single-run evaluations will not match the paper's numbers.
• The 80:20 train/test split is performed randomly without explicit stratification, which can cause severe class imbalance in 1-shot or 5-shot settings.

Evidence (verbatim from paper)

Here, we use classification accuracy and macro F1 scores as the performance metrics.

Citation

@misc{schiavone2023myriadal,
  title={MyriadAL: Active Few Shot Learning for Histopathology},
  author={Schiavone et al. (2023)},
  year={2023},
  note={arXiv:2310.16161}
}

• arXiv: 2310.16161