Qata Cov19 Eval

qata-cov19-eval

COVID-19 Infection Map Generation and Detection from Chest X-Ray Images — Degerli et al. (2020) (arXiv:2009.12698, 2020)

What this evaluates

Evaluates deep learning models for COVID-19 infected region segmentation and binary detection on chest X-ray images. It probes the model's ability to localize pathological regions at the pixel level and classify whole images as positive or negative for infection.

Datasets

• QaTa-COV19 — total ?; splits: Group-I (15495), Group-II (-1)

Metrics

• F1-Score (primary) — range: percent
  • Harmonic mean of Precision and Sensitivity: F1 = 2 * (Precision * Sensitivity) / (Precision + Sensitivity).
• Sensitivity — range: percent
  • Recall: TP / (TP + FN).
• Specificity — range: percent
  • TN / (TN + FP).
• Precision — range: percent
  • TP / (TP + FP).
• Accuracy — range: percent
  • (TP + TN) / (TP + TN + FP + FN).
• F2-Score — range: percent
  • Weighted F-score emphasizing FN minimization: F2 = 5 * (Precision * Sensitivity) / (4 * Precision + Sensitivity).

Input / output format

Input: Chest X-ray (CXR) images resized to 224x224 pixels.

Output: For segmentation: pixel-level binary mask (infected region vs background). For detection: binary class label (COVID-19 positive vs control negative).

Scoring recipe

def compute_metrics(tp, tn, fp, fn):
    sensitivity = tp / (tp + fn)
    specificity = tn / (tn + fp)
    precision = tp / (tp + fp)
    accuracy = (tp + tn) / (tp + tn + fp + fn)
    f1 = 2 * precision * sensitivity / (precision + sensitivity)
    f2 = 5 * precision * sensitivity / (4 * precision + sensitivity)
    return sensitivity, specificity, precision, accuracy, f1, f2

Common pitfalls

• Significant class imbalance between COVID-19 and control samples requires explicit data augmentation (shifting, rotation) to balance training sets.
• Performance is reported as mean ± 95% confidence interval over 5-fold cross-validation, not single-run accuracy.
• Encoder layers may be frozen or unfrozen during training, which drastically changes the reported metric values.

Evidence (verbatim from paper)

The standard performance evaluation metrics are defined as follows: Sensitivity = TP/(TP+FN), Specificity = TN/(TN+FP), Precision = TP/(TP+FP), Accuracy = (TP+TN)/(TP+TN+FP+FN), F(β) = (1+β²)(Precision×Sensitivity)/(β²×Precision+Sensitivity). The F1-Score is calculated with β=1, which is the harmonic average of precision and sensitivity. The F2-score is calculated with β=2, which emphasizes FN minimization over FPs.

Citation

@misc{degerli2020covid19infectionmap,
  title={COVID-19 Infection Map Generation and Detection from Chest X-Ray Images},
  author={Degerli et al. (2020)},
  year={2020},
  note={arXiv:2009.12698}
}

• arXiv: 2009.12698