Early Qata Cov19 Eval

early-qata-cov19-eval

Advance Warning Methodologies for COVID-19 using Chest X-Ray Images — Ahishali et al. (2020) (arXiv:2006.05332, 2020)

What this evaluates

Evaluates machine learning models' ability to detect early-stage COVID-19 infection from chest X-ray images, specifically targeting cases with minimal or invisible radiological signs compared to healthy controls.

Datasets

• Early-QaTa-COV19 — total 13609; splits: train (10887), test (2722); repo https://github.com/metahishali/methods-early-cov19

Metrics

• sensitivity (primary) — range: percent
  • True Positive Rate: proportion of actual early-stage COVID-19 cases correctly identified by the model.
• specificity — range: percent
  • True Negative Rate: proportion of normal/control cases correctly identified by the model.

Input / output format

Input: Chest X-ray images resized to 224×224 pixels.

Output: Binary class prediction (Early Stage COVID-19 vs Normal) or class probabilities via SoftMax.

Scoring recipe

# 5-fold cross-validation evaluation
sensitivity_scores = []
specificity_scores = []
for fold in range(5):
    train_X, train_y = get_fold(fold, split='train')
    test_X, test_y = get_fold(fold, split='test')
    model = train_model(train_X, train_y)
    preds = model.predict(test_X)
    tp = sum((preds == 1) & (test_y == 1))
    fn = sum((preds == 0) & (test_y == 1))
    tn = sum((preds == 0) & (test_y == 0))
    fp = sum((preds == 1) & (test_y == 0))
    sensitivity_scores.append(tp / (tp + fn))
    specificity_scores.append(tn / (tn + fp))
mean_sensitivity = sum(sensitivity_scores) / 5
mean_specificity = sum(specificity_scores) / 5

Common pitfalls

• Dataset is highly imbalanced (1:12 ratio), requiring explicit balancing via augmentation or class weights during training.
• Early-stage labels are assigned via visual inspection rather than strict temporal criteria, introducing potential label noise.
• High intra-class dissimilarity stems from multi-source compilation, which may degrade out-of-distribution generalization.

Evidence (verbatim from paper)

The comparative methods are evaluated by a 5-fold cross-validation (CV) scheme over the Early-QaTa-COV19 dataset. We have resized chest X-ray images to 224 × 224 in order to fit the input dimensions to the state-of-the-art deep network topologies. Table 1 shows the number of samples in each fold, which we split the data into training and test (unseen folds) sets by 80% and 20%, respectively.

Citation

@misc{ahishali2020advance,
  title={Advance Warning Methodologies for COVID-19 using Chest X-Ray Images},
  author={Ahishali et al. (2020)},
  year={2020},
  note={arXiv:2006.05332}
}

• arXiv: 2006.05332