qhjqhj00 ptbxl-ecg-classification-eval
Self-Supervised Pre-Training with Joint-Embedding Predictive Architecture Boosts ECG Classification Performance — Weimann et al. (2024) (arXiv:2410.13867, 2024)
What this evaluates
Evaluates the ability of self-supervised pre-trained Vision Transformers to classify ECG signals across multiple diagnostic label hierarchies (e.g., all statements, ST-MEM labels, diagnostic subclasses, rhythm statements).
Datasets
- PTB-XL — total ?; splits: train (-1), val (-1), test (-1); HF
physionet/ptb-xl
Metrics
macro AUC(primary) — range: [0, 1]- Computes the Area Under the Receiver Operating Characteristic (AUC) curve for each diagnostic label independently, then averages these AUC values across all K labels.
Input / output format
Input: ECG time-series waveforms (typically 12-lead, processed as image-like patches/tokens for Vision Transformer input).
Output: Binary classification probabilities or scores for each diagnostic label in the target hierarchy.
Scoring recipe
def compute_macro_auc(y_true, y_pred, n_labels):
auc_scores = []
for i in range(n_labels):
auc_scores.append(roc_auc_score(y_true[:, i], y_pred[:, i]))
return np.mean(auc_scores)
Common pitfalls
- Linear evaluation (frozen backbone + linear head) frequently outperforms end-to-end fine-tuning for JEPA-pretrained ViTs on PTB-XL.
- The ST-MEM baseline comparison uses a custom data split, making performance comparisons approximate rather than strictly controlled.
- Best downstream performance does not always correlate with lowest pre-training loss; checkpoint selection must be based on validation AUC rather than pre-training loss.
Evidence (verbatim from paper)
Performance is evaluated using the macro area under the receiver operating curve (macro AUC), the primary metric that is universally reported in related studies. We report the average macro AUC on the test set (fold 10) with the standard deviation in parentheses (0.0xx).
Citation
@misc{weimann2024selfsupervised,
title={Self-Supervised Pre-Training with Joint-Embedding Predictive Architecture Boosts ECG Classification Performance},
author={Weimann et al. (2024)},
year={2024},
note={arXiv:2410.13867}
}
- arXiv: 2410.13867