Enginead Eval

enginead-eval

EngineAD: A Real-World Vehicle Engine Anomaly Detection Dataset — Hojjati et al. (2026) (arXiv:2603.25955, 2026)

What this evaluates

Probes the ability of one-class anomaly detection algorithms to identify incipient engine faults in real-world, multivariate vehicle sensor telemetry. It specifically evaluates cross-vehicle generalization and robustness to distributional shifts in normal operating conditions across a commercial fleet.

Datasets

• EngineAD — total ?; splits: train (-1), test (-1); repo https://github.com/Armanfard-Lab/EngineAD

Metrics

• F1-score (anomaly class) (primary) — range: [0, 1]
  • Harmonic mean of precision and recall computed exclusively on the anomaly class (positive class). Designed to balance false positives and false negatives in highly imbalanced fault detection scenarios.

Input / output format

Input: Flattened 1D feature vector of shape (2400,) representing a 300-timestep multivariate time-series segment across 8 principal components, standardized using a scaler fitted solely on training data.

Output: Binary classification label (0=normal, 1=anomaly) derived from model anomaly scores, typically thresholded at the 95th percentile of training normal scores for scoring-based methods.

Scoring recipe

def compute_f1_anomaly(y_true, y_pred):
    tp = np.sum((y_true == 1) & (y_pred == 1))
    fp = np.sum((y_true == 0) & (y_pred == 1))
    fn = np.sum((y_true == 1) & (y_pred == 0))
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    return f1

Common pitfalls

• The test set is not a simple random split; it concatenates 30% of normal segments with all available anomaly segments, creating an artificial class balance that differs from the raw data distribution.
• Thresholds for distance/scoring methods are fixed at the 95th percentile of the training normal data scores, not tuned on the test set, which heavily constrains the precision/recall trade-off.
• Input segments are flattened into 1D vectors, discarding temporal structure, which may unfairly disadvantage sequence-aware or recurrent models compared to classical point-wise detectors.

Evidence (verbatim from paper)

Performance was measured using the F1-score calculated specifically for the anomaly class, as this metric provides a robust assessment of a detector’s efficacy in highly imbalanced and critical tasks.

Citation

@misc{hojjati2026enginead,
  title={EngineAD: A Real-World Vehicle Engine Anomaly Detection Dataset},
  author={Hojjati et al. (2026)},
  year={2026},
  note={arXiv:2603.25955}
}

• arXiv: 2603.25955