Chest Xray14 Eval

chest-xray14-eval

Weighing Features of Lung and Heart Regions for Thoracic Disease Classification — Fang et al. (2021) (arXiv:2105.12430, 2021)

What this evaluates

Evaluates a model's ability to perform multi-label classification of 14 thoracic diseases on chest X-ray images and localize pathological regions using attention maps. It probes whether anatomically grounded feature weighting improves disease detection over global feature fusion or saliency-based methods.

Datasets

• Chest X-ray14 — total ?; splits: benchmark split (-1), box set (-1)

Metrics

• AUROC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, computed per disease class across varying classification thresholds to measure the trade-off between true positive and false positive rates.
• IoU — range: [0, 1]
  • Intersection over Union between the predicted pathological region (derived from Class Activation Maps) and the ground-truth bounding box.

Input / output format

Input: Chest X-ray radiograph images.

Output: Per-image class probabilities for 14 thoracic diseases, and predicted bounding boxes for pathological regions via Class Activation Maps (CAM).

Scoring recipe

def compute_auroc(y_true, y_pred):
    fpr, tpr, _ = roc_curve(y_true, y_pred)
    return auc(fpr, tpr)

def compute_iou(pred_box, gt_box):
    intersection = area(pred_box & gt_box)
    union = area(pred_box | gt_box)
    return intersection / union if union > 0 else 0.0

# Per-disease AUROC
aurocs = [compute_auroc(gold[d], preds[d]) for d in diseases]
avg_auroc = mean(aurocs)

# Localization IoU
ious = [compute_iou(get_cam_box(img), img.gt_box) for img in box_set]
avg_iou = mean(ious)

Common pitfalls

• Relying solely on global image features causes local discriminative information to be smoothed out.
• Methods using saliency maps or region proposals suffer from location deviation in pathological regions.
• Small pathological regions (e.g., nodules) are easily drowned out by global features without explicit feature weighting.

Evidence (verbatim from paper)

In Table 2, we report the classification performances of the proposed method and comparative methods in terms of AUROC scores, evaluated by the test set of the benchmark split.

Citation

@misc{fang2021weighing,
  title={Weighing Features of Lung and Heart Regions for Thoracic Disease Classification},
  author={Fang et al. (2021)},
  year={2021},
  note={arXiv:2105.12430}
}

• arXiv: 2105.12430