Ptbxl Multi Label Ecg Eval

ptbxl-multi-label-ecg-eval

How Much Temporal Modeling is Enough? A Systematic Study of Hybrid CNN-RNN Architectures for Multi-Label ECG Classification — Jafari et al. (2026) (arXiv:2601.18830, 2026)

What this evaluates

Evaluates deep learning architectures for multi-label classification of 12-lead ECG recordings into 23 diagnostic categories. Probes the trade-off between local morphological feature extraction and sequential temporal modeling under severe class imbalance.

Datasets

• PTB-XL — total ?; splits: test (-1); repo https://github.com/alireza720/Multi-Label-Classification

Metrics

• Macro AUROC (primary) — range: [0, 1]
  • Area under the receiver operating characteristic curve, averaged across all 23 diagnostic classes. Computed by calculating the AUROC for each class independently and taking the unweighted mean.
• Hamming Loss — range: [0, 1]
  • Fraction of label assignments that are incorrect. Calculated as the number of misclassified labels divided by the total number of labels (N × C).
• Subset Accuracy — range: [0, 1]
  • Fraction of samples for which all predicted labels exactly match all true labels (exact match ratio).

Input / output format

Input: Single 12-lead ECG time-series recording.

Output: Binary multi-label vector of length 23, where each element indicates the presence (1) or absence (0) of a specific diagnostic class.

Scoring recipe

def evaluate(y_true, y_pred):
    # y_true, y_pred: (N, 23) binary matrices
    hamming_loss = np.mean(y_true != y_pred)
    subset_acc = np.mean(np.all(y_true == y_pred, axis=1))
    macro_auroc = np.mean([roc_auc_score(y_true[:, c], y_pred[:, c]) for c in range(23)])
    macro_f1 = np.mean([f1_score(y_true[:, c], y_pred[:, c], average='macro') for c in range(23)])
    return {'hamming_loss': hamming_loss, 'subset_accuracy': subset_acc, 'macro_auroc': macro_auroc, 'macro_f1': macro_f1}

Common pitfalls

• Threshold-dependent metrics (F1, AUPRC) are highly unstable for rare classes due to extreme prevalence skew, often showing near-perfect precision but low recall.
• Deeper or hybrid recurrent stacks (e.g., LSTM+BiLSTM) yield diminishing returns and may overfit, making simpler single-layer BiLSTM or CNN architectures more robust.
• High AUROC for minority classes can mask poor threshold-based predictive performance, misleadingly suggesting clinical utility for rare diagnoses.

Evidence (verbatim from paper)

Among the evaluated models, the BiLSTM model attains the lowest Hamming Loss (0.0338) and the highest Subset Accuracy (0.5723), reflecting fewer label-wise errors and improved joint label assignment in the multi-label setting. It also demonstrates competitive macro- and micro-averaged performance, including Macro AUROC (0.9202), Macro AUPRC (0.4715), and Micro F1 (0.6979), supporting its effectiveness in capturing temporal dependencies relevant to ECG interpretation.

Citation

@misc{jafari2026temporal,
  title={How Much Temporal Modeling is Enough? A Systematic Study of Hybrid CNN-RNN Architectures for Multi-Label ECG Classification},
  author={Jafari et al. (2026)},
  year={2026},
  note={arXiv:2601.18830}
}

• arXiv: 2601.18830