Mit Bih Ecg Adv Detection Eval

mit-bih-ecg-adv-detection-eval

ECG-Adv-GAN: Detecting ECG Adversarial Examples with Conditional Generative Adversarial Networks — Hossain et al. (2021) (arXiv:2107.07677, 2021)

What this evaluates

Evaluates the robustness of ECG arrhythmia classifiers and adversarial detectors on real and synthetically generated adversarial ECG signals. It probes whether models maintain classification accuracy and can distinguish between genuine and adversarial cardiac signals under intra-patient and inter-patient data splits.

Datasets

• PhysioNet MIT-BIH Arrhythmia dataset — total ?; splits: train (-1), test (-1), intra-patient (-1), inter-patient (-1)

Metrics

• Accuracy (ACC) (primary) — range: percent
  • Ratio of correctly classified instances to the total number of instances.
• Sensitivity (SEN) — range: percent
  • True positive rate: proportion of actual positives correctly identified.
• Specificity (SPEC) — range: percent
  • True negative rate: proportion of actual negatives correctly identified.
• F1 Score — range: [0, 1]
  • Harmonic mean of precision and recall.
• AUC — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, measuring discriminative ability across thresholds.
• MSE — range: other
  • Mean Squared Error between original and adversarial signal amplitudes.
• SSIM — range: [0, 1]
  • Structural Similarity Index measuring perceived change in structural information.

Input / output format

Input: Normalized ECG signal segments (lead II) of 280 samples centered on the R-peak, with amplitude scaled to [0,1].

Output: Class label (N, S, V, or F) for arrhythmia classification; binary label (Real or Adversarial) for detection.

Scoring recipe

def compute_metrics(y_true, y_pred):
    tp = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 1)
    tn = sum(1 for t, p in zip(y_true, y_pred) if t == 0 and p == 0)
    fp = sum(1 for t, p in zip(y_true, y_pred) if t == 0 and p == 1)
    fn = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 0)
    acc = (tp + tn) / (tp + tn + fp + fn)
    sen = tp / (tp + fn) if (tp + fn) > 0 else 0
    spec = tn / (tn + fp) if (tn + fp) > 0 else 0
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    f1 = 2 * prec * sen / (prec + sen) if (prec + sen) > 0 else 0
    return acc, sen, spec, f1

Common pitfalls

• Class imbalance in minority classes (S, V, F) requires SMOTE oversampling during training, which may not reflect real-world deployment.
• Baselines lack pre-trained weights, necessitating full retraining on the exact same splits for fair comparison.
• Adversarial examples are generated by the proposed model itself, potentially biasing detection evaluation toward the generator's specific perturbation style.

Evidence (verbatim from paper)

For metrics, we use Accuracy (ACC), Sensitivity(SEN), and Specificity(SPEC). We can see for the first experiment, except for the architecture given in [[3]], our model achieves the best score compared to other deep learning and machine learning derived architectures.

Citation

@misc{hossain2021ecgadv_gan,
  title={ECG-Adv-GAN: Detecting ECG Adversarial Examples with Conditional Generative Adversarial Networks},
  author={Hossain et al. (2021)},
  year={2021},
  note={arXiv:2107.07677}
}

• arXiv: 2107.07677