Hydraulic Anomaly Detection Eval

hydraulic-anomaly-detection-eval

Comparative Study on Semi-supervised Learning Applied for Anomaly Detection in Hydraulic Condition Monitoring System — Yongqi Dong et al. (2023) (arXiv:2306.02709, 2023)

What this evaluates

Evaluates semi-supervised and traditional machine learning models for detecting hydraulic system anomalies using only normal data for training. It probes the ability of models to generalize from normal-condition features and identify leakage faults under class-imbalanced testing conditions.

Datasets

• Unspecified hydraulic condition monitoring dataset — total ?; splits: train (-1), valid (-1), test (-1)

Metrics

• ACC — range: [0, 1]
  • Number of correctly classified instances divided by the total number of instances.
• TPR — range: [0, 1]
  • True Positives divided by the sum of True Positives and False Negatives (Recall).
• FPR — range: [0, 1]
  • False Positives divided by the sum of False Positives and True Negatives.
• F1_Score (primary) — range: [0, 1]
  • Harmonic mean of precision and recall: 2 * (Precision * Recall) / (Precision + Recall).

Input / output format

Input: Feature vectors representing hydraulic system operational states (normal vs. anomalous/leakage conditions).

Output: Binary classification label (normal or anomaly) or anomaly score used with a decision threshold.

Scoring recipe

def compute_metrics(y_true, y_pred):
    tp = sum((y_true == 1) & (y_pred == 1))
    tn = sum((y_true == 0) & (y_pred == 0))
    fp = sum((y_true == 0) & (y_pred == 1))
    fn = sum((y_true == 1) & (y_pred == 0))
    acc = (tp + tn) / (tp + tn + fp + fn)
    tpr = tp / (tp + fn) if (tp + fn) > 0 else 0
    fpr = fp / (fp + tn) if (fp + tn) > 0 else 0
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    f1 = 2 * precision * (tp / (tp + fn)) / (precision + (tp / (tp + fn))) if (precision + (tp / (tp + fn))) > 0 else 0
    return acc, tpr, fpr, f1

Common pitfalls

• Training and validation sets contain exclusively normal data, requiring semi-supervised or one-class learning approaches rather than standard supervised training.
• The test set is heavily imbalanced with a majority of anomalies, which can make accuracy metrics misleading without careful threshold tuning.
• Deep autoencoder models underperform traditional methods in this domain, indicating that feature engineering or model architecture choice is critical.

Evidence (verbatim from paper)

To test the selected semi-supervised models, the dataset was split into the train, valid, and test sets, with train and valid sets only containing normal data samples, and the majority of the test set being anomalies. Table III synthesizes the quantitative performance comparison results of the selected semi-supervised ML models. As shown in the table, HELM provides the best performance obtaining the highest accuracy (99.5%), the lowest false positive rate (0.015), and the best F1-score (0.985) beating other semi-supervised methods.

Citation

@misc{dong2023hydraulic,
  title={Comparative Study on Semi-supervised Learning Applied for Anomaly Detection in Hydraulic Condition Monitoring System},
  author={Yongqi Dong et al. (2023)},
  year={2023},
  note={arXiv:2306.02709}
}

• arXiv: 2306.02709