Mit Bih Ecg Classification Eval

mit-bih-ecg-classification-eval

Real-Time Patient-Specific ECG Classification by 1D Self-Operational Neural Networks — Malik et al. (2021) (arXiv:2110.02215, 2021)

What this evaluates

Evaluates a model's ability to classify individual ECG beats into standard AAMI categories (Normal, Supraventricular Ectopic, Ventricular Ectopic, etc.) on a patient-specific basis. It probes robustness to severe class imbalance and morphological variations in real-time clinical monitoring scenarios.

Datasets

• MIT-BIH arrhythmia database — total 100389; splits: test (-1)

Metrics

• F1-score (primary) — range: [0, 1] or percent
  • Harmonic mean of precision and recall: F1 = 2 * (Precision * Recall) / (Precision + Recall). Computed per class and averaged for multi-class or binary detection tasks.
• Accuracy — range: [0, 1] or percent
  • Ratio of correctly classified beats to total beats: (TP + TN) / Total.
• Sensitivity — range: [0, 1] or percent
  • True positive rate: TP / (TP + FN).
• Specificity — range: [0, 1] or percent
  • True negative rate: TN / (TN + FP).
• Positive Predictivity — range: [0, 1] or percent
  • Precision: TP / (TP + FP).

Input / output format

Input: Single or dual-channel raw ECG beat segments resized to 128 samples.

Output: Discrete class label per beat (N, S, V, F, Q) or binary SVEB/VEB detection.

Scoring recipe

def compute_metrics(preds, gold, target):
    tp = sum(1 for p, g in zip(preds, gold) if p == g == target)
    fp = sum(1 for p, g in zip(preds, gold) if p == target and g != target)
    fn = sum(1 for p, g in zip(preds, gold) if p != target and g == target)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    accuracy = sum(1 for p, g in zip(preds, gold) if p == g) / len(gold)
    return accuracy, f1, precision, recall

Common pitfalls

• Patient-specific partitioning violates standard i.i.d. assumptions; models are trained on a small common set plus patient-specific data, making cross-patient generalization evaluation fundamentally different from standard benchmarks.
• Severe class imbalance exists (e.g., ~39k Normal vs ~190 SVEB beats), so Accuracy alone is misleading; F1 and Sensitivity must be prioritized.
• Some competing methods use non-AAMI compliant data partitioning (e.g., training on data from 200 patients), which inflates performance unfairly compared to the strict patient-specific protocol.

Evidence (verbatim from paper)

In this study, the following standard metrics are used: classification accuracy (Acc), sensitivity (Sen), specificity (Spe), positive predictivity (Ppr), and F1-score (F1). Since there is a large variation in the number of beats from different classes (class imbalance) in the training/testing data (i.e. 39465/50354 type-N, 1277/5716 type-V, and 190/2571 type-S beats), sensitivity, specificity, positive predictive value and especially, F1-score are all relevant performance criteria for medical diagnosis applications.

Citation

@misc{malik2021realtime,
  title={Real-Time Patient-Specific ECG Classification by 1D Self-Operational Neural Networks},
  author={Malik et al. (2021)},
  year={2021},
  note={arXiv:2110.02215}
}

• arXiv: 2110.02215