Covid19 Cxr Ct Detection Eval

covid19-cxr-ct-detection-eval

Explainable and Lightweight Model for COVID-19 Detection Using Chest Radiology Images — Suba S et al. (2022) (arXiv:2212.13788, 2022)

What this evaluates

Evaluates a lightweight CNN's ability to classify chest radiology images (CXR and CT scans) as positive or negative for COVID-19, and in a three-class setting. It also tests cross-modality generalization by training on CT and testing on CXR data.

Datasets

• CXR/CT Chest Radiology Dataset — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/aleesuss/c19

Metrics

• accuracy (primary) — range: [0, 1]
  • TP / TP +0.5*(FP + FN) as defined in the paper (note: non-standard formulation).
• precision — range: [0, 1]
  • TP / (TP + FP)
• recall — range: [0, 1]
  • TP / (TP + FN)
• f1-score — range: [0, 1]
  • Calculated per class separately using precision and recall.

Input / output format

Input: Chest X-ray (CXR) or Chest CT images resized to 224x224x3 pixels.

Output: Binary classification (sigmoid activation) or three-class classification (softmax activation) indicating COVID-19 status.

Scoring recipe

def compute_metrics(y_true, y_pred):
    tp = np.sum((y_true == 1) & (y_pred == 1))
    fp = np.sum((y_true == 0) & (y_pred == 1))
    fn = np.sum((y_true == 1) & (y_pred == 0))
    accuracy = tp / (tp + 0.5 * (fp + fn))
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    return accuracy, precision, recall, f1

Common pitfalls

• The paper defines accuracy as TP / (TP + 0.5*(FP + FN)), which deviates from the standard TP/(TP+FP+FN) formula.
• F1-score is reported per class separately rather than as a macro/micro average, which can obscure overall performance on imbalanced datasets.
• Cross-modality evaluation (training on CT, testing on CXR) is mentioned but not fully detailed in the results section, making reproducibility of that specific protocol difficult.

Evidence (verbatim from paper)

Performance comparison is carried out for various parameters, viz., accuracy, precision, recall, F1-score, time taken to train the model, number of model parameters, etc. Precision, recall and F1-score are reported per class and accuracy is computed considering all classes. Accuracy is calculated as TP / TP +0.5*(FP + FN), where TP, FP and FN correspond to the number of true positives, false positives and false negatives, respectively. Recall (Sensitivity) gives the proportion of correct positive results and is defined as the ratio of true positives to all positive cases, TP/(TP+FN). Precision (Positive Predictive Value - PPV) gives the proportion of positively classified cases that are truly positive: TP/(TP+FP) and is a measure of how relevant a positive result is. F1-score is calculated the same way as accuracy but for each class separately.

Citation

@misc{suba2022explainable,
  title={Explainable and Lightweight Model for COVID-19 Detection Using Chest Radiology Images},
  author={Suba S et al. (2022)},
  year={2022},
  note={arXiv:2212.13788}
}

• arXiv: 2212.13788