Ecg Robustness Eval

ecg-robustness-eval

ECG-ATK-GAN: Robustness against Adversarial Attacks on ECGs using Conditional Generative Adversarial Networks — Hossain et al. (2021) (arXiv:2110.09983, 2021)

What this evaluates

Evaluates the robustness of ECG classification models against six adversarial attack types (FGSM, BIM, PGD, CW, DBB, HSJ) compared to clean data. It measures classification performance and signal generation quality on two public ECG datasets.

Datasets

• PhysioNet MIT-BIH Arrhythmia — total ?; splits: train (-1), test (-1)
• PTB Diagnostic ECG Database — total ?; splits: train (-1), test (-1)

Metrics

• Accuracy (primary) — range: [0, 1]
  • Proportion of correctly classified instances (clean or attacked) out of the total test set instances.
• Sensitivity — range: [0, 1]
  • True positive rate: proportion of actual positive instances correctly identified.
• Specificity — range: [0, 1]
  • True negative rate: proportion of actual negative instances correctly identified.
• Structural Similarity (SSIM) — range: [0, 1]
  • Perceptual similarity metric between generated and real ECG signals, ranging from 0 to 1.
• Mean-Squared-Error (MSE) — range: other
  • Average squared difference between generated and real signal amplitudes.
• Normalized Mean Squared Error (NRMSE) — range: [0, 1]
  • MSE normalized by the range or mean of the signal to provide scale-invariant error measurement.
• Cross-correlation Coefficient — range: other
  • Measures linear correlation between generated and real signals across time lags.

Input / output format

Input: ECG signal segments centered on R-peaks, sampled at 280 Hz, normalized to [0,1]. Inputs include both clean beats and adversarially perturbed versions (FGSM, BIM, PGD, CW, DBB, HSJ).

Output: Predicted class labels (N, S, V, F for MIT-BIH; Normal, Myocardial Infarction for PTB) and attack type classification (clean vs. specific attack).

Scoring recipe

def compute_metrics(preds, labels):
    tp = sum(1 for p, l in zip(preds, labels) if p == l and l == 1)
    tn = sum(1 for p, l in zip(preds, labels) if p == l and l == 0)
    fp = sum(1 for p, l in zip(preds, labels) if p == 1 and l == 0)
    fn = sum(1 for p, l in zip(preds, labels) if p == 0 and l == 1)
    accuracy = (tp + tn) / (tp + tn + fp + fn)
    sensitivity = tp / (tp + fn) if (tp + fn) > 0 else 0
    specificity = tn / (tn + fp) if (tn + fp) > 0 else 0
    return accuracy, sensitivity, specificity

Common pitfalls

• SMOTE oversampling is applied only to the training set, not the test set, meaning test performance reflects the original imbalanced distribution.
• HSJ attacks introduce high-frequency noise that obscures signal patterns, causing disproportionate performance drops across all models and potentially skewing average robustness scores.
• Models are trained on a combined dataset of clean and attacked signals, but evaluated separately on clean vs. attacked test splits, requiring strict separation to avoid data leakage.

Evidence (verbatim from paper)

For metrics, we use Accuracy, Sensitivity, and Specificity. We can see that for ‘No Attack’, all models achieve comparatively good results. However, for each distinct attack, the results worsen for other models compared to ours.

Citation

@misc{hossain2021ecgatk,
  title={ECG-ATK-GAN: Robustness against Adversarial Attacks on ECGs using Conditional Generative Adversarial Networks},
  author={Hossain et al. (2021)},
  year={2021},
  note={arXiv:2110.09983}
}

• arXiv: 2110.09983