Chest Xray Classification Eval

chest-xray-classification-eval

Advancing Diagnostic Precision: Leveraging Machine Learning Techniques for Accurate Detection of Covid-19, Pneumonia, and Tuberculosis in Chest X-Ray Images — Kulkarni et al. (2023) (arXiv:2310.06080, 2023)

What this evaluates

Evaluates a CNN's ability to classify chest X-ray images into disease categories (COVID-19, pneumonia, tuberculosis, normal) using various preprocessing techniques. It probes robustness across different dataset sizes and class distributions.

Datasets

• Multiclass Chest X-ray Dataset — total 7135; splits: test (-1)
• Hamad Medical Corporation Tuberculosis Dataset — total 6300; splits: test (-1)
• Pneumonia Dataset — total 5863; splits: test (-1)
• NIH Chest X-ray Dataset — total 112120; splits: test (-1)

Metrics

• AUC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve. Measures the trade-off between true positive rate and false positive rate across classification thresholds.
• F1-Score — range: [0, 1]
  • Harmonic mean of precision and recall: 2 * (Precision * Recall) / (Precision + Recall).
• Precision — range: [0, 1]
  • True Positives / (True Positives + False Positives).
• Recall — range: [0, 1]
  • True Positives / (True Positives + False Negatives).

Input / output format

Input: Preprocessed chest X-ray images (adaptive thresholding, LTP, histogram equalization, augmentation).

Output: Class label prediction (e.g., 'Covid-19', 'Normal', 'Pneumonia', 'Tuberculosis') or probability scores for AUC calculation.

Scoring recipe

def compute_metrics(y_true, y_pred, y_prob):
    tp = np.sum((y_true == 1) & (y_pred == 1))
    fp = np.sum((y_true == 0) & (y_pred == 1))
    fn = np.sum((y_true == 1) & (y_pred == 0))
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    auc = roc_auc_score(y_true, y_prob)
    return precision, recall, f1, auc

Common pitfalls

• Class imbalance heavily impacts COVID-19 recall due to limited images.
• AUC is reported per class but macro/micro averaging method is unspecified.
• No explicit train/validation/test split methodology is described; results appear to be from a single evaluation run.
• Preprocessing choice drastically alters performance, making cross-study comparisons difficult.

Evidence (verbatim from paper)

The proposed approach is evaluated as follows: 1. The network is tested on the multiclass classification dataset consisting of 7135 chest X-ray images, along with a comparison with other pre-trained models. 2. The network is tested on the tuberculosis dataset from the Hamad Medical Corporation containing 6300 chest X-ray images. 3. The network is also tested on the Pneumonia dataset containing 5863 chest X-ray images. 4. The network is evaluated on the benchmark NIH dataset. The network performs with a strikingly good AUC value of 0.99 and a precision value of 0.9799 for the tuberculosis class.

Citation

@misc{kulkarni2023advancing,
  title={Advancing Diagnostic Precision: Leveraging Machine Learning Techniques for Accurate Detection of Covid-19, Pneumonia, and Tuberculosis in Chest X-Ray Images},
  author={Kulkarni et al. (2023)},
  year={2023},
  note={arXiv:2310.06080}
}

• arXiv: 2310.06080