Dinids Cross Domain Eval

dinids-cross-domain-eval

DI-NIDS: Domain Invariant Network Intrusion Detection System — Layeghy et al. (2022) (arXiv:2210.08252, 2022)

What this evaluates

This benchmark evaluates the robustness of network intrusion detection models against distribution shifts between different network environments. It specifically probes cross-domain generalization by training on one NetFlow dataset and testing on another, measuring how well domain-invariant feature extraction mitigates performance degradation when facing unseen attack distributions.

Datasets

• NFv2-UNSW-NB15 — total 2390275; splits: train (-1), test (-1)
• NFv2-CIC-2018 — total 18893708; splits: train (-1), test (-1)

Metrics

• F1-Score (primary) — range: percent
  • Harmonic mean of precision and recall for binary classification (Benign vs. Attack). Calculated as 2 * (precision * recall) / (precision + recall). Reported as a percentage in the paper.

Input / output format

Input: 43 NetFlow version 9 fields representing bi-directional network flows, with binary labels (Benign or Attack).

Output: Binary prediction indicating whether a network flow is Benign or Attack.

Scoring recipe

def compute_f1_score(preds, gold):
    tp = sum(1 for p, g in zip(preds, gold) if p == 1 and g == 1)
    fp = sum(1 for p, g in zip(preds, gold) if p == 1 and g == 0)
    fn = sum(1 for p, g in zip(preds, gold) if p == 0 and g == 1)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    return f1 * 100

Common pitfalls

• Cross-domain evaluation direction drastically affects baseline performance; reporting only one direction (e.g., CIC→UNSW) hides severe degradation in the reverse direction.
• Aggregating diverse attack types into a single 'Attack' class changes the anomaly detection task from multi-class classification to binary anomaly detection, which may not reflect real-world multi-attack scenarios.

Evidence (verbatim from paper)

Table 4 shows the results (F1-Score) of domain-specific performance evaluation for DI-NIDS, and the six baseline ML models used for comparison.

Citation

@misc{layeghy2022dinids,
  title={DI-NIDS: Domain Invariant Network Intrusion Detection System},
  author={Layeghy et al. (2022)},
  year={2022},
  note={arXiv:2210.08252}
}

• arXiv: 2210.08252