Netguard Nids Eval

netguard-nids-eval

Generative Active Adaptation for Drifting and Imbalanced Network Intrusion Detection — Ragini Gupta et al. (2025) (arXiv:2503.03022, 2025)

What this evaluates

Evaluates a generative active adaptation framework for network intrusion detection under concept drift and class imbalance. It probes the model's ability to select informative samples, generate synthetic minority-class data, and maintain high detection performance across shifting temporal and spatial domains with limited labeling budgets.

Datasets

• CIC-IDS (2017/2018) — total 100000; splits: train (70000), test (30000)
• UGR'16 — total 100000; splits: train (70000), test (30000)

Metrics

• F1-score (primary) — range: percent
  • Harmonic mean of precision and recall: 2 * (Precision * Recall) / (Precision + Recall). Reported as a percentage across all classes and per rare class.
• FPR — range: [0, 1]
  • False Positive Rate: FP / (FP + TN). Measures the proportion of actual benign flows incorrectly classified as attacks.
• FNR — range: [0, 1]
  • False Negative Rate: FN / (TP + FN). Measures the proportion of actual attacks incorrectly classified as benign.
• Accuracy — range: percent
  • Proportion of correctly classified instances: (TP + TN) / Total.
• AUROC — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve. Measures the model's ability to distinguish between classes across all classification thresholds.

Input / output format

Input: Tabular network flow feature vectors (e.g., packet counts, protocol types, duration, flags) from CIC-IDS or UGR'16 datasets.

Output: Predicted attack class label (e.g., Benign, Bot, DDoS, DoS variants, FTP-BruteForce, Infiltration, SSH-Bruteforce, Web Attack) and confidence scores for uncertainty-based sampling.

Scoring recipe

from sklearn.metrics import f1_score, accuracy_score
def score(y_true, y_pred):
    f1 = f1_score(y_true, y_pred, average='macro') * 100
    acc = accuracy_score(y_true, y_pred) * 100
    return {'F1': f1, 'Accuracy': acc}

Common pitfalls

• The labeling budget (0.1%, 0.5%, 1%) applies strictly to the test dataset size, not the training set.
• The 70/30 split is used for both source and target domains in bidirectional drift scenarios (2017→2018 and 2018→2017), which fundamentally changes the distribution shift direction.
• Synthetic augmentation is applied only to specific rare classes (DoS Slowhttptest, FTP-BruteForce, Infiltration, Web Attack) at 3x/10x ratios, not uniformly across all classes.

Evidence (verbatim from paper)

We evaluate NetGuard end-to-end performance (including prior selection and data augmentation) based on F1-score improvement under drift, FPR, FNR, Accuracy, and rare class F1 improvement.

Citation

@misc{gupta2025generative,
  title={Generative Active Adaptation for Drifting and Imbalanced Network Intrusion Detection},
  author={Ragini Gupta et al. (2025)},
  year={2025},
  note={arXiv:2503.03022}
}

• arXiv: 2503.03022