Unsw Nb15 Nids Eval

unsw-nb15-nids-eval

A Low-Cost Machine Learning Based Network Intrusion Detection System with Data Privacy Preservation — Fakirah et al. (2021) (arXiv:2107.02362, 2021)

What this evaluates

Evaluates the classification accuracy and computational efficiency of machine learning models for network intrusion detection on a realistic dataset of contemporary traffic and synthetic attacks. It also assesses the privacy preservation and data utility of a Pearson Correlation Coefficient (PCC) feature selection and Least Squares Method (LSM) data distortion pipeline.

Datasets

• UNSW-NB15 — total 257673; splits: train (175341), test (82332)

Metrics

• Accuracy (primary) — range: [0, 1]
  • Accuracy = (TP + TN) / (TP + TN + FP + FN). Measures the proportion of correctly classified normal and attack records.
• Precision, Recall, F-Score, Specificity — range: [0, 1]
  • Precision = TP/(TP+FP); Recall = TP/(TP+FN); F-Score = 2*(P*R)/(P+R); Specificity = TN/(TN+FP). Standard binary classification metrics reported alongside Accuracy.
• Privacy Measures (VD, RP, RK, CP, CK) — range: other
  • Value Difference (VD), Rank Position (RP), Rank Maintenance (RK), Change of Rank (CP), and Maintenance of Rank (CK) quantify distortion between original and transformed datasets. Formulas are defined in reference [16].

Input / output format

Input: Network flow records represented as 44 numerical/categorical features across Flow, Basic, Content, Time, and Additional General categories, plus a binary label (0: normal, 1: attack). After PCC filtering, 27 features are used as input.

Output: Predicted binary label (0 or 1) per network flow record.

Scoring recipe

tp = sum(p == 1 and g == 1 for p, g in zip(preds, golds))
tn = sum(p == 0 and g == 0 for p, g in zip(preds, golds))
fp = sum(p == 1 and g == 0 for p, g in zip(preds, golds))
fn = sum(p == 0 and g == 1 for p, g in zip(preds, golds))
accuracy = (tp + tn) / (tp + tn + fp + fn)
precision = tp / (tp + fp) if (tp + fp) > 0 else 0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0
f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
specificity = tn / (tn + fp) if (tn + fp) > 0 else 0

Common pitfalls

• The paper applies a PCC correlation threshold of 0.85 to drop features before training, which alters input dimensionality and must be replicated exactly to match reported runtimes and accuracy.
• Privacy metrics (VD, RP, RK, CP, CK) are computed on transformed vs. original data using formulas defined in reference [16], not standard ML metrics, making direct replication difficult without the cited source.
• Categorical features ('proto', 'state', 'service') are explicitly dropped during PCC analysis due to level differences between train/test sets, which affects feature alignment and model training.

Evidence (verbatim from paper)

The performance of the proposed PCC-LSM-NIDS is gauged using the conventional True Positive (TP), False Negative (FN), False Positive (FP) and True Negative (TN) measures [15]. ... Specificity, Precision, Accuracy and F-Score values, except Recall. ... Data Utility (DU) and Privacy Measures (PMs) are two important metrics to evaluate the efficiency of the proposed PCC-LSM-NIDS. ... The relative value difference between the original and transformed dataset is obtained using the value difference (VD). To quantify the change in value positions, rank position (RP), rank maintenance (RK), change of rank of features (CP), and maintenance of rank of features (CK) are computed.

Citation

@misc{fakirah2021lowcost,
  title={A Low-Cost Machine Learning Based Network Intrusion Detection System with Data Privacy Preservation},
  author={Fakirah et al. (2021)},
  year={2021},
  note={arXiv:2107.02362}
}

• arXiv: 2107.02362