Chestxray14 Plco Eval

chestxray14-plco-eval

Learning to recognize Abnormalities in Chest X-Rays with Location-Aware Dense Networks — Gündel et al. (2018) (arXiv:1803.04565, 2018)

What this evaluates

Evaluates a model's ability to detect and localize multiple pathologies in high-resolution chest X-ray images. It specifically probes the model's robustness to severe class imbalance and its capacity to leverage explicit spatial location information for pathology classification.

Datasets

• ChestX-Ray14 — total ?; splits: train (-1), val (-1), test (-1)
• PLCO — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• AUC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, computed independently for each pathology class and typically averaged across all classes to report overall performance.

Input / output format

Input: 1024x1024 RGB chest X-ray images, normalized to ImageNet mean and standard deviation.

Output: C-dimensional vector of probabilities in [0,1] via sigmoid, where C=14 for ChestX-Ray14 or C=35 for the combined dataset, representing independent binary pathology predictions.

Scoring recipe

def compute_auc(predictions, labels):
    # predictions: (N, C) probabilities from sigmoid
    # labels: (N, C) binary ground truth
    auc_scores = []
    for c in range(C):
        auc_scores.append(roc_auc_score(labels[:, c], predictions[:, c]))
    return np.mean(auc_scores)

Common pitfalls

• Using image-wise splits instead of patient-wise splits causes data leakage and artificially inflates performance.
• Failing to account for severe class imbalance without appropriate loss weighting or thresholding leads to biased predictions.
• Treating spatial location labels as mutually exclusive when multiple diseases can co-occur in different lung regions.

Evidence (verbatim from paper)

achieving the highest AUC on ChestX-Ray14 under fair evaluation conditions

Citation

@misc{gundel2018dnetloc,
  title={Learning to recognize Abnormalities in Chest X-Rays with Location-Aware Dense Networks},
  author={Gündel et al. (2018)},
  year={2018},
  note={arXiv:1803.04565}
}

• arXiv: 1803.04565