Osbad Eval

osbad-eval

An Open-Access Benchmark of Statistical and Machine-Learning Anomaly Detection Methods for Battery Applications — Mei-Chin Pang et al. (2025) (arXiv:2511.01745, 2025)

What this evaluates

Evaluates the ability of statistical and machine learning models to detect anomalies in battery discharge capacity profiles across different chemistries. It probes cross-chemistry generalization and model robustness on imbalanced, rare-anomaly datasets typical of electrochemical systems.

Datasets

• MIT/Stanford (Severson) — total 46; splits: train (23), test (23); repo https://github.com/meichinpang/osbad
• Tohoku — total 10; splits: train (4), test (4); repo https://github.com/meichinpang/osbad

Metrics

• AUROC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across all classification thresholds.

Input / output format

Input: Discharge cycle profiles (voltage, current, capacity) for each battery cell.

Output: Binary anomaly label per cycle (0 = inlier/normal, 1 = outlier/anomalous).

Scoring recipe

def compute_auroc(y_true, y_scores):
    from sklearn.metrics import roc_auc_score
    return roc_auc_score(y_true, y_scores)

Common pitfalls

• Data leakage if test cells are used during hyperparameter tuning or baseline assessment
• AUROC can be misleading on highly imbalanced datasets with rare anomalies
• Cross-chemistry generalization requires evaluating models trained on liquid electrolyte cells against solid electrolyte test cells

Evidence (verbatim from paper)

The labeled test set is kept completely separate until the final evaluation to prevent data leakage and inflated performance estimates. The framework highlights the limitations of traditional metrics (e.g., AUROC) in imbalanced, rare-anomaly settings.

Citation

@misc{pang2025osbad,
  title={An Open-Access Benchmark of Statistical and Machine-Learning Anomaly Detection Methods for Battery Applications},
  author={Mei-Chin Pang et al. (2025)},
  year={2025},
  note={arXiv:2511.01745}
}

• arXiv: 2511.01745