Mit Bih Arrhythmia Eval

mit-bih-arrhythmia-eval

ECG-Based Heart Arrhythmia Diagnosis Through Attentional Convolutional Neural Networks — Liu et al. (2021) (arXiv:2108.10226, 2021)

What this evaluates

Evaluates a model's ability to classify raw 2-lead ECG signals into five arrhythmia types (normal, supraventricular, ventricular, fusion, unknown) using a multi-class classification setup. It probes temporal feature extraction and attention-based weighting of cardiac cycles without manual preprocessing.

Datasets

• MIT-BIH Arrhythmia Dataset — total 96000; splits: train (76800), test (19200); repo https://github.com/ziyuliu-lion/heart-arrhythmia-diagnosis-with-deep-learning

Metrics

• AUC (primary) — range: [0.5, 1]
  • Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across classification thresholds.
• F-1 score — range: [0, 1]
  • Harmonic mean of precision and recall: 2 * (precision * recall) / (precision + recall). Reported per class and averaged to handle severe class imbalance.

Input / output format

Input: 2-channel ECG segment of 240 sampling points per channel (approx. 0.76 seconds), centered on the R-peak, with z-score normalization applied.

Output: Predicted class label or probability distribution over 5 arrhythmia categories: N (normal), S (supraventricular), V (ventricular), F (fusion), Q (unknown).

Scoring recipe

import numpy as np
from sklearn.metrics import roc_auc_score, f1_score

# AUC (One-vs-Rest for multi-class)
auc = roc_auc_score(y_true, y_pred_proba, multi_class='ovr', average='macro')

# F-1 score (Weighted average to handle class imbalance)
f1 = f1_score(y_true, y_pred, average='weighted')

Common pitfalls

• Class imbalance is severe (e.g., F=154 vs N=16884), making accuracy and macro-averaged precision/recall misleading; AUC and weighted F-1 are explicitly preferred.
• The 80/20 split is random across all 96,000 samples, not stratified by patient; this risks data leakage if subject-level cross-validation is expected.
• Input windows are strictly fixed to 240 samples centered on the R-peak; models trained on variable-length or non-aligned segments will not match the reported protocol.

Evidence (verbatim from paper)

The AUC score falls in the range of $[0.5,1]$, the higher the better. Although the commonly used evaluation metrics (such as accuracy, precision, recall, and F-1 score) are not suitable as they all sensitive the sample amount, we use them to show the model’s performance on each single class.

Citation

@misc{liu2021ecgarrhythmia,
  title={ECG-Based Heart Arrhythmia Diagnosis Through Attentional Convolutional Neural Networks},
  author={Liu et al. (2021)},
  year={2021},
  note={arXiv:2108.10226}
}

• arXiv: 2108.10226