Dy Meter Eval

dy-meter-eval

Catching Every Ripple: Enhanced Anomaly Awareness via Dynamic Concept Adaptation — Zhu et al. (2026) (arXiv:2604.14726, 2026)

What this evaluates

Evaluates online anomaly detection models under concept drift by testing their ability to adapt to evolving data distributions without retraining. It probes instance-level sensitivity to context-dependent anomalies across continuous and discrete streaming scenarios.

Datasets

• Ionosphere — total 351; splits: stream (-1)
• Pima — total 768; splits: stream (-1)
• Satellite — total 6435; splits: stream (-1)
• Mammography — total 11183; splits: stream (-1)
• BGL — total 4713493; splits: stream (-1)
• NSL-KDD — total 125973; splits: stream (-1)
• KDD99 — total 494021; splits: stream (-1)
• Activity Recognition — total 6675; splits: stream (-1)
• Internal Bleeding — total 7492; splits: stream (-1)
• NASA — total 11299; splits: stream (-1)
• GaitPhase — total 11991; splits: stream (-1)
• EPG — total 30000; splits: stream (-1)
• ECG — total 80000; splits: stream (-1)
• Machine temperature — total 22695; splits: stream (-1)
• CPU utilization — total 18050; splits: stream (-1)
• INSECTS-Abr — total 44569; splits: stream (-1)
• INSECTS-Inc — total 48086; splits: stream (-1)
• INSECTS-IncGrd — total 20367; splits: stream (-1)
• INSECTS-IncRec — total 67455; splits: stream (-1)
• SynM-AbrRec — total 20480; splits: stream (-1)
• SynM-GrdRec — total 20480; splits: stream (-1)
• SynF-AbrRec — total 20480; splits: stream (-1)
• SynF-GrdRec — total 20480; splits: stream (-1)

Metrics

• AUCROC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, plotting False Positive Rate against True Positive Rate across all classification thresholds.
• AUCPR — range: [0, 1]
  • Area under the Precision-Recall curve, displaying the relationship between precision and recall at differing thresholds.

Input / output format

Input: Sequential feature vectors arriving in a data stream, processed in sliding windows of size 64. Data may have temporal dependencies (continuous setting) or be independent (discrete setting).

Output: Continuous anomaly score per instance, used to compute threshold-independent curves.

Scoring recipe

import numpy as np
from sklearn.metrics import roc_auc_score, average_precision_score

def compute_metrics(y_true, scores):
    auc_roc = roc_auc_score(y_true, scores)
    auc_pr = average_precision_score(y_true, scores)
    return {'AUCROC': auc_roc, 'AUCPR': auc_pr}

Common pitfalls

• Datasets are evaluated in a streaming/online setting with sliding windows, not standard static train/test splits.
• Synthetic datasets simulate concept drift by randomly swapping MNIST/FMNIST categories as anomaly targets, which differs from standard image benchmarks.
• Performance depends on searched hyperparameters (e.g., pseudo-labeling threshold μ_p ∈ [0.05, 0.5], uncertainty threshold μ_e ∈ [0.005, 0.4]), which must be tuned per dataset.

Evidence (verbatim from paper)

We utilize AUCROC and AUCPR metrics for evaluation. AUCROC calculates the area under the receiver operating characteristic (ROC) curve, plotting the false positive rate (FPR) against the true positive rate (TPR) across various thresholds. AUCPR measures the area under the precision-recall (PR) curve, displaying the relationship between precision and recall at differing thresholds. Both metrics range from 0 to 1, with higher values indicating superior performance.

Citation

@misc{zhu2026catching,
  title={Catching Every Ripple: Enhanced Anomaly Awareness via Dynamic Concept Adaptation},
  author={Zhu et al. (2026)},
  year={2026},
  note={arXiv:2604.14726}
}

• arXiv: 2604.14726