Mobilecaps Covidx Eval

mobilecaps-covidx-eval

MobileCaps: A Lightweight Model for Screening and Severity Analysis of COVID-19 Chest X-Ray Images — Pawan et al. (2021) (arXiv:2108.08775, 2021)

What this evaluates

Evaluates a lightweight hybrid deep learning model for classifying chest X-ray images into COVID-19, Normal, and Pneumonia categories, and optionally predicting disease severity scores.

Datasets

• COVIDx — total 15477; splits: train (13898), test (1579); repo https://github.com/ActiveNeuron/MobileCaps

Metrics

• F1 Score (primary) — range: percent
  • Harmonic mean of precision and recall: F1 = 2 * (Precision * Recall) / (Precision + Recall). Reported per class (COVID-19, Normal, Pneumonia) and averaged.
• Precision — range: percent
  • Ratio of true positives to all predicted positives: TP / (TP + FP).
• Recall — range: percent
  • Ratio of true positives to all actual positives: TP / (TP + FN).

Input / output format

Input: Chest X-ray (CXR) images resized to 224x224x3 pixels.

Output: Classification probabilities for three classes (COVID-19, Normal, Pneumonia) in range [0, 1], or a mapped severity score (1-8) for the MobileCaps-S variant.

Scoring recipe

def compute_metrics(predictions, gold):
    tp = sum(1 for p, g in zip(predictions, gold) if p == g == 1)
    fp = sum(1 for p, g in zip(predictions, gold) if p == 1 and g == 0)
    fn = sum(1 for p, g in zip(predictions, gold) if p == 0 and g == 1)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    return precision, recall, f1

Common pitfalls

• Metrics are reported per class (C, N, P) rather than as a single macro-average, which can obscure overall performance on imbalanced splits.
• External validation datasets (Twitter, JSS) are strictly held-out test sets; models are not fine-tuned on them, limiting direct comparison with models trained on external data.

Evidence (verbatim from paper)

Table 1 shows the quantitative results depicting the superiority of the proposed architecture. Capsule Network and MobileNetV2 achieved an F1 score of 74.03 and 82.90, respectively whereas, the proposed method achieved 94.91, which is remarkable.

Citation

@misc{pawan2021mobilecaps,
  title={MobileCaps: A Lightweight Model for Screening and Severity Analysis of COVID-19 Chest X-Ray Images},
  author={Pawan et al. (2021)},
  year={2021},
  note={arXiv:2108.08775}
}

• arXiv: 2108.08775