Covid19 Xray Classification Eval

covid19-xray-classification-eval

CT-to-X-ray Distillation Under Tiny Paired Cohorts: An Evidence-Bounded Reproducible Pilot Study — Bo Ma et al. (2026) (arXiv:2603.29167, 2026)

What this evaluates

This evaluation probes a model's ability to classify chest X-rays as COVID-19 positive or negative using a cross-modal distillation setup where CT images are only used during training. It specifically tests the robustness of transfer learning under extremely small, patient-level paired cohorts and prevalence-heavy validation splits.

Datasets

• COVID-19 Image Data Collection — total 783; splits: train (628), val (155)

Metrics

• Accuracy (primary) — range: [0, 1]
  • Fraction of correctly classified instances out of the total validation set. Computed as (TP+TN)/(TP+TN+FP+FN).
• Macro-F1 — range: [0, 1]
  • Unweighted mean of recall or precision for each class, averaged across the two classes (COVID-19 and non-COVID).
• Balanced Accuracy — range: [0, 1]
  • Average of recall obtained on each class, computed as (TPR+TNR)/2, which mitigates class imbalance effects.
• Specificity — range: [0, 1]
  • True negative rate, calculated as TN/(TN+FP). Measures the proportion of actual negatives correctly identified.
• MCC — range: [-1, 1]
  • Matthews Correlation Coefficient, a correlation coefficient between observed and predicted binary classifications. Ranges from -1 to +1, with +1 representing perfect prediction.
• PR-AUC — range: [0, 1]
  • Area under the Precision-Recall curve, computed by integrating precision over recall thresholds. Particularly informative for imbalanced datasets.

Input / output format

Input: 128×128 grayscale chest X-ray images. For cross-modal experiments, CT images are provided only during teacher training, not at inference.

Output: Binary class label (COVID-19 vs. non-COVID) or probability scores for the positive class.

Scoring recipe

import numpy as np
from sklearn.metrics import accuracy_score, f1_score, balanced_accuracy_score, specificity_score, matthews_corrcoef, precision_recall_curve, auc

def compute_metrics(y_true, y_pred, y_prob=None):
    acc = accuracy_score(y_true, y_pred)
    macro_f1 = f1_score(y_true, y_pred, average='macro')
    bal_acc = balanced_accuracy_score(y_true, y_pred)
    spec = specificity_score(y_true, y_pred)
    mcc = matthews_corrcoef(y_true, y_pred)
    if y_prob is not None:
        prec, rec, _ = precision_recall_curve(y_true, y_prob)
        pr_auc = auc(rec, prec)
    else:
        pr_auc = np.nan
    return {'Accuracy': acc, 'Macro-F1': macro_f1, 'Balanced Accuracy': bal_acc,
            'Specificity': spec, 'MCC': mcc, 'PR-AUC': pr_auc}

Common pitfalls

• Validation splits are extremely small (4 images in the fixed split, 5–10 per Monte Carlo resample), making bootstrap confidence intervals and paired significance tests numerically unreliable.
• Splits are heavily prevalence-heavy with only one negative validation patient per resample, which artificially inflates accuracy and necessitates balanced accuracy, specificity, and MCC for fair assessment.
• Results are highly unstable across random seeds and resampling runs; single-run rankings or mean differences should not be interpreted as definitive performance claims.

Evidence (verbatim from paper)

The headline table intentionally keeps one non-paired reference row and then separates the shared paired fixed split below it. That paired fixed split contains only four validation patients and four X-ray images (three positive, one negative), so we keep the main-text table focused on accuracy, macro-F1, and balanced accuracy rather than on threshold-sensitive secondary metrics.

Citation

@misc{ma2026cttoxraydistillation,
  title={CT-to-X-ray Distillation Under Tiny Paired Cohorts: An Evidence-Bounded Reproducible Pilot Study},
  author={Bo Ma et al. (2026)},
  year={2026},
  note={arXiv:2603.29167}
}

• arXiv: 2603.29167