Pdcovidnet Eval

pdcovidnet-eval

PDCOVIDNet: A Parallel-Dilated Convolutional Neural Network Architecture for Detecting COVID-19 from Chest X-Ray Images — Chowdhury et al. (2020) (arXiv:2007.14777, 2020)

What this evaluates

Evaluates a CNN's ability to classify chest X-ray images into three diagnostic categories: COVID-19, Normal, and Viral Pneumonia. It probes multi-scale feature extraction and robustness to class imbalance in medical imaging.

Datasets

• Custom benchmark dataset — total 2905; splits: train (-1), val (-1), test (-1)

Metrics

• Accuracy (primary) — range: percent
  • Ratio of correctly classified samples to total samples: (TP + TN) / Total Samples.
• F1 — range: percent
  • Harmonic mean of Precision and Recall: 2 * (Precision * Recall) / (Precision + Recall).
• AUC — range: [0, 1]
  • Area under the Receiver Operating Characteristic (ROC) curve, measuring the trade-off between true positive rate and false positive rate across classification thresholds.

Input / output format

Input: Chest X-ray images

Output: Softmax probability distribution over three classes: COVID-19, Normal, Viral Pneumonia

Scoring recipe

def compute_metrics(y_true, y_pred, y_prob):
    tp = np.sum((y_true == 1) & (y_pred == 1))
    tn = np.sum((y_true == 0) & (y_pred == 0))
    fp = np.sum((y_true == 0) & (y_pred == 1))
    fn = np.sum((y_true == 1) & (y_pred == 0))
    acc = (tp + tn) / (tp + tn + fp + fn)
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
    auc = roc_auc_score(y_true, y_prob)
    return acc, prec, rec, f1, auc

Common pitfalls

• High class imbalance (219 COVID-19 vs 2686 non-COVID images) makes Accuracy misleading; F1/AUC are emphasized as more reliable.
• Confusion matrix analysis references a test set of only 23 images, contradicting the stated 10% split (~290 images), suggesting a potential subset evaluation or reporting inconsistency.
• Models are trained from scratch without ImageNet pre-training, which is non-standard for CNNs and limits direct comparison to architectures typically evaluated with transfer learning.

Evidence (verbatim from paper)

For experimental evaluations, we utilized several evaluation metrics such as Accuracy, Precision, Recall, and F1 score, i.e., ... where $TP$ stands for true positive, while $TN$, $FP$, and $FN$ stand for true negative, false positive, and false negative, respectively. The $F1$ score may be a more reliable measure because the benchmark dataset is unbalanced, such as COVID-19 with $219$ images and non-COVID with $2686$ images. Subsequently, we used the ROC (Receiver Operating Characteristics) curve to display the results and measured the area under the ROC curve (often called AUC (Area Under the Curve)) to provide information about the effectiveness of the model.

Citation

@misc{chowdhury2020pdcovidnet,
  title={PDCOVIDNet: A Parallel-Dilated Convolutional Neural Network Architecture for Detecting COVID-19 from Chest X-Ray Images},
  author={Chowdhury et al. (2020)},
  year={2020},
  note={arXiv:2007.14777}
}

• arXiv: 2007.14777