Wsn Intrusion Detection Eval

wsn-intrusion-detection-eval

MLSTL-WSN: Machine Learning-based Intrusion Detection using SMOTETomek in WSNs — Talukder et al. (2024) (arXiv:2402.13277, 2024)

What this evaluates

Evaluates machine learning classifiers for binary and multiclass intrusion detection in Wireless Sensor Networks (WSNs), specifically testing their robustness on imbalanced datasets and the impact of SMOTETomek resampling.

Datasets

• Large-scale WSN dataset — total 374661; splits: test (-1)

Metrics

• Accuracy (primary) — range: percent
  • Calculated as (TP + TN) / (TP + FP + FN + TN), representing the proportion of correctly classified instances out of the total.
• Precision — range: percent
  • Calculated as TP / (TP + FP), measuring the proportion of positive predictions that are actually correct.
• Recall — range: percent
  • Calculated as TP / (TP + FN), measuring the proportion of actual positives that are correctly identified.
• F1-Score — range: percent
  • Harmonic mean of Precision and Recall: 2 * (Precision * Recall) / (Precision + Recall).
• MAE — range: other
  • Mean Absolute Error: average of absolute differences between predicted and actual values.
• MSE — range: other
  • Mean Squared Error: average of squared differences between predicted and actual values.
• RMSE — range: other
  • Root Mean Squared Error: square root of MSE.
• AUC — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve, measuring the model's ability to distinguish between classes across all thresholds.

Input / output format

Input: Numerical feature vectors representing WSN network traffic/sensor data, standardized and optionally resampled via SMOTETomek.

Output: Binary label (normal/intrusion) or multiclass label (specific intrusion type).

Scoring recipe

def compute_metrics(y_true, y_pred):
    tp = np.sum((y_true == 1) & (y_pred == 1))
    tn = np.sum((y_true == 0) & (y_pred == 0))
    fp = np.sum((y_true == 0) & (y_pred == 1))
    fn = np.sum((y_true == 1) & (y_pred == 0))
    accuracy = (tp + tn) / (tp + tn + fp + fn)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    mae = np.mean(np.abs(y_true - y_pred))
    mse = np.mean((y_true - y_pred) ** 2)
    rmse = np.sqrt(mse)
    auc = roc_auc_score(y_true, y_pred)
    return accuracy, precision, recall, f1, mae, mse, rmse, auc

Common pitfalls

• Resampling (SMOTETomek) applied before train/test split causes data leakage and inflates metrics.
• High overall accuracy on imbalanced data may mask poor detection of minority intrusion classes.
• MAE formula in the paper omits absolute value bars in the numerator, which could lead to sign cancellation if not implemented correctly.

Evidence (verbatim from paper)

The evaluation of our proposed model involved the utilization of multiple performance metrics to assess its effectiveness. These metrics are defined as follows: - Confusion Matrix: Table 5 shows the confusion matrix where TP represents True Positive, TN represents True Negative, FP stands for False Positive, and FN denotes False Negative. Accuracy: $$ Accuracy = rac {T P + T N}{T P + F P + F N + T N} $$ - Precision: $$ Precision = rac {T P}{T P + F P} $$ - Recall: $$ ext {R e c a l l} = rac {T P}{T P + F N} $$

Citation

@misc{talukder2024mlstlwsn,
  title={MLSTL-WSN: Machine Learning-based Intrusion Detection using SMOTETomek in WSNs},
  author={Talukder et al. (2024)},
  year={2024},
  note={arXiv:2402.13277}
}

• arXiv: 2402.13277