# Ptbxl Ecg Eval

> Evaluates the ability of foundation models to perform multi-label classification on clinical 12-lead ECG recordings. It probes robustness to class imbalance and sample efficiency by measuring performance across varying label sets and training data sizes. Use when the user wants to benchmark on PTB-XL, or asks about evaluating this task. Reports macro AUROC.

- Skill: `qhjqhj00/ptbxl-ecg-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/ptbxl-ecg-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/ptbxl-ecg-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/ptbxl-ecg-eval

---


# ptbxl-ecg-eval

> Bridging Performance Gaps for Foundation Models: A Post-Training Strategy for ECGFounder — Zhou et al. (arXiv:2509.12991, 2025)

## What this evaluates

Evaluates the ability of foundation models to perform multi-label classification on clinical 12-lead ECG recordings. It probes robustness to class imbalance and sample efficiency by measuring performance across varying label sets and training data sizes.

## Datasets

- **PTB-XL** — total 21837; splits: train (-1), val (-1), test (-1)

## Metrics

- `macro AUROC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, averaged across all labels (macro-averaged). Computed per label and then averaged.
- `macro AUPRC` — range: [0, 1]
  - Area under the Precision-Recall Curve, averaged across all labels (macro-averaged). More sensitive to class imbalance than AUROC.

## Input / output format

**Input**: 10-second clinical 12-lead ECG recordings

**Output**: Multi-label probability scores or binary predictions for up to 71 ECG statements (rhythm, diagnostic, or subclass categories)

## Scoring recipe

```python
def compute_macro_auc_pr(y_true, y_pred, num_labels):
    aucs, prcs = [], []
    for i in range(num_labels):
        aucs.append(roc_auc_score(y_true[:, i], y_pred[:, i]))
        prcs.append(auc_precision_recall_curve(y_true[:, i], y_pred[:, i]))
    return np.mean(aucs), np.mean(prcs)
```

## Common pitfalls

- AUROC can be misleadingly high on highly imbalanced ECG datasets; AUPRC is recommended for imbalanced scenarios.
- Multiple labels can be assigned to a single recording, requiring multi-label evaluation rather than single-label accuracy.
- Performance confidence intervals are derived from 1000 bootstrapped samples, not standard deviation across folds.

## Evidence (verbatim from paper)

> Table 1 presents the macro AUROC of all methods, along with 95% confidence intervals computed using 1000 bootstrapped samples. Although AUROC is a standard benchmark metric, it may be less sensitive in the presence of class imbalance. To address this, we additionally report macro Area Under the Precision-Recall Curve (AUPRC) in Table [2]...

## Citation

```bibtex
@misc{zhou2025bridging,
  title={Bridging Performance Gaps for Foundation Models: A Post-Training Strategy for ECGFounder},
  author={Zhou et al.},
  year={2025},
  note={arXiv:2509.12991}
}
```

- arXiv: 2509.12991