Label Noise Resilience Histo Eval

label-noise-resilience-histo-eval

Contrastive-Based Deep Embeddings for Label Noise-Resilient Histopathology Image Classification — Dedieu et al. (2024) (arXiv:2404.07605, 2024)

What this evaluates

This benchmark evaluates the robustness of histopathology image classification models to both uniform and asymmetric label noise. It compares the performance of contrastive deep embeddings against non-contrastive backbones and image-based noise-robust loss functions. The protocol measures how well classifiers maintain accuracy when training labels are corrupted.

Datasets

• NCT-CRC-HE-100K — total ?; splits: train (80000), val (20000), test (7180)
• PatchCamelyon — total ?; splits: train (262144), val (32768), test (32768)
• BACH — total ?; splits: train (320), val (80), test (83)
• MHIST — total ?; splits: train (1740), val (435), test (977)
• LC25000 — total ?; splits: train (16000), val (4000), test (5000)
• GasHisSDB — total ?; splits: train (21303), val (5325), test (6656)

Metrics

• test accuracy (primary) — range: percent
  • Percentage of correctly predicted labels out of the total number of test instances. Reported as mean ± standard deviation across multiple independent training runs.

Input / output format

Input: Histopathology image patches of varying resolutions (e.g., 224×224, 96×96, 2048×1536, 768×768, 160×160) with class labels subject to uniform or asymmetric noise injection during training.

Output: Predicted class label for each image patch.

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100
# Report mean ± std over 2 runs (image methods) or 4 runs (deep embedding methods)

Common pitfalls

• Noise is applied during the training phase to the labels used for model fitting, not merely at test time.
• Standard deviation is computed over multiple independent training runs (2 for image-based baselines, 4 for deep embedding methods), not across different data splits.
• Asymmetric noise uses dataset-specific class transition matrices rather than a simple uniform flip rate.

Evidence (verbatim from paper)

Table 2: Average test accuracies with standard deviation under different uniform label noise ratios (%, 2 runs for image methods, 4 runs for deep embedding methods).

Citation

@misc{dedieu2024contrastive,
  title={Contrastive-Based Deep Embeddings for Label Noise-Resilient Histopathology Image Classification},
  author={Dedieu et al. (2024)},
  year={2024},
  note={arXiv:2404.07605}
}

• arXiv: 2404.07605