Ecg Ssl Eval

ecg-ssl-eval

NERULA: A Dual-Pathway Self-Supervised Learning Framework for Electrocardiogram Signal Analysis — Manimaran et al. (2024) (arXiv:2405.19348, 2024)

What this evaluates

Evaluates the transferability of self-supervised representations learned from single-lead ECG signals to downstream clinical and activity recognition tasks. It probes the model's ability to extract robust cardiac and physiological features by training linear probes on frozen encoder outputs across classification and regression benchmarks.

Datasets

• PhysioNet 2017 — total 8528; splits: train (-1), val (-1)
• PTB-XL — total ?; splits: train (-1), val (-1), test (-1)
• Human Activity Recognition (HAR) — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• Accuracy (primary) — range: [0, 1]
  • Proportion of correctly predicted class labels out of the total number of instances. Computed for arrhythmia, gender, and activity recognition tasks.
• F1 Score — range: [0, 1]
  • Harmonic mean of precision and recall, typically calculated with macro averaging to handle class imbalance in multi-class arrhythmia detection.
• AUC — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, measuring the probability that a randomly chosen positive instance ranks higher than a randomly chosen negative instance.
• MAE — range: other
  • Mean Absolute Error, representing the average magnitude of errors between predicted and true continuous values without considering direction.
• R² — range: other
  • Coefficient of determination, indicating the proportion of variance in the target variable (age) that is predictable from the model features.

Input / output format

Input: Single-lead ECG time-series signals (lead-II extracted from 12-lead recordings) or multi-axis IMU sensor data (accelerometer/gyroscope). Pre-trained encoder extracts fixed-length feature vectors, which are concatenated per axis for HAR tasks before being fed to linear probes.

Output: Predicted class labels (arrhythmia, gender, activity) or continuous values (age). Performance is measured by training linear classifiers/regressors (SVC, Random Forest, Logistic Regression, MLP) on frozen representations.

Scoring recipe

def compute_metrics(y_true, y_pred, y_proba, task_type):
    if task_type == 'classification':
        acc = accuracy_score(y_true, y_pred)
        f1 = f1_score(y_true, y_pred, average='macro')
        auc = roc_auc_score(y_true, y_proba, multi_class='ovr')
        return {'Accuracy': acc, 'F1 Score': f1, 'AUC': auc}
    elif task_type == 'regression':
        mae = mean_absolute_error(y_true, y_pred)
        r2 = r2_score(y_true, y_pred)
        return {'MAE': mae, 'R²': r2}

Common pitfalls

• Using multiple ECG leads instead of restricting to lead-II/single-lead as explicitly specified in the protocol.
• Using ground-truth clinical labels during the self-supervised pre-training phase, which violates the SSL setup.
• Ignoring the preset train/validation split for PhysioNet 2017 and creating arbitrary splits for linear probing.

Evidence (verbatim from paper)

This dataset consists of 8,528 recordings and has a preset split of train and validation, which we use to both fit the machine learning model (SVC, Random Forest, and Logistic Regression) and get the scores(Accuracy, F1, AUC).

Citation

@misc{manimaran2024nerula,
  title={NERULA: A Dual-Pathway Self-Supervised Learning Framework for Electrocardiogram Signal Analysis},
  author={Manimaran et al. (2024)},
  year={2024},
  note={arXiv:2405.19348}
}

• arXiv: 2405.19348