Ptbxl Ecg Eval

ptbxl-ecg-eval

Bridging Performance Gaps for Foundation Models: A Post-Training Strategy for ECGFounder — Zhou et al. (arXiv:2509.12991, 2025)

What this evaluates

Evaluates the ability of foundation models to perform multi-label classification on clinical 12-lead ECG recordings. It probes robustness to class imbalance and sample efficiency by measuring performance across varying label sets and training data sizes.

Datasets

• PTB-XL — total 21837; splits: train (-1), val (-1), test (-1)

Metrics

• macro AUROC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, averaged across all labels (macro-averaged). Computed per label and then averaged.
• macro AUPRC — range: [0, 1]
  • Area under the Precision-Recall Curve, averaged across all labels (macro-averaged). More sensitive to class imbalance than AUROC.

Input / output format

Input: 10-second clinical 12-lead ECG recordings

Output: Multi-label probability scores or binary predictions for up to 71 ECG statements (rhythm, diagnostic, or subclass categories)

Scoring recipe

def compute_macro_auc_pr(y_true, y_pred, num_labels):
    aucs, prcs = [], []
    for i in range(num_labels):
        aucs.append(roc_auc_score(y_true[:, i], y_pred[:, i]))
        prcs.append(auc_precision_recall_curve(y_true[:, i], y_pred[:, i]))
    return np.mean(aucs), np.mean(prcs)

Common pitfalls

• AUROC can be misleadingly high on highly imbalanced ECG datasets; AUPRC is recommended for imbalanced scenarios.
• Multiple labels can be assigned to a single recording, requiring multi-label evaluation rather than single-label accuracy.
• Performance confidence intervals are derived from 1000 bootstrapped samples, not standard deviation across folds.

Evidence (verbatim from paper)

Table 1 presents the macro AUROC of all methods, along with 95% confidence intervals computed using 1000 bootstrapped samples. Although AUROC is a standard benchmark metric, it may be less sensitive in the presence of class imbalance. To address this, we additionally report macro Area Under the Precision-Recall Curve (AUPRC) in Table [2]...

Citation

@misc{zhou2025bridging,
  title={Bridging Performance Gaps for Foundation Models: A Post-Training Strategy for ECGFounder},
  author={Zhou et al.},
  year={2025},
  note={arXiv:2509.12991}
}

• arXiv: 2509.12991