Lunet Ids Eval

lunet-ids-eval

LuNet: A Deep Neural Network for Network Intrusion Detection — Wu et al. (2019) (arXiv:1909.10031, 2019)

What this evaluates

Evaluates a deep neural network's capability to classify network traffic packets as normal or specific attack types. It probes spatial-temporal feature extraction, handling of class imbalance, and robustness against overlapping attack signatures in intrusion detection systems.

Datasets

• NSL-KDD — total ?; splits: test (-1)
• UNSW-NB15 — total ?; splits: test (-1)

Metrics

• Detection Rate (DR%) (primary) — range: percent
  • Percentage of actual attacks correctly identified. Calculated as True Positives divided by the sum of True Positives and False Negatives, multiplied by 100.
• Accuracy (ACC%) — range: percent
  • Percentage of all correctly classified packets (both normal and attack) out of the total number of packets evaluated.
• False Positive Rate (FPR%) — range: percent
  • Percentage of normal traffic incorrectly classified as attacks. Calculated as False Positives divided by the sum of False Positives and True Negatives, multiplied by 100.

Input / output format

Input: Feature vectors representing network traffic packets, processed through hierarchical CNN and RNN subnets to capture spatial and temporal patterns.

Output: Discrete class label indicating either 'normal' traffic or a specific attack category (5 classes for NSL-KDD, 10 classes for UNSW-NB15).

Scoring recipe

def compute_metrics(y_true, y_pred):
    correct = sum(1 for t, p in zip(y_true, y_pred) if t == p)
    accuracy = (correct / len(y_true)) * 100
    # DR and FPR are computed per attack class and macro-averaged across K-Fold splits
    return accuracy

Common pitfalls

• The NSL-KDD dataset is highly imbalanced; relying solely on raw accuracy can be misleading, which is why the authors use Stratified K-Fold Cross Validation.
• Rare attack categories (e.g., U2R, R2L, Backdoor, Worms) consistently show low detection rates due to significant feature overlap with other attacks and insufficient training samples (~1-1.4% of the dataset).

Evidence (verbatim from paper)

Table I shows the detection rate, accuracy and false positive rate for the binary classification of LuNet under different Stratified K-Fold Cross Validations, with k ranging from 2 to 10. The average values are given in the last row of the table.

Citation

@misc{wu2019lunet,
  title={LuNet: A Deep Neural Network for Network Intrusion Detection},
  author={Wu et al. (2019)},
  year={2019},
  note={arXiv:1909.10031}
}

• arXiv: 1909.10031