Kdd99 Ids Eval

kdd99-ids-eval

An Implementation of Intrusion Detection System Using Genetic Algorithm — Hoque et al. (2012) (arXiv:1204.1336, 2012)

What this evaluates

Evaluates an intrusion detection system's ability to classify network traffic into normal and specific attack categories (probe, dos, u2r, r2l) using genetic algorithm-optimized feature selection and rule generation.

Datasets

• KDD99 — total ?; splits: test (-1)

Metrics

• Detection Rate (DR) (primary) — range: [0, 1]
  • Ratio of correctly detected intrusions to the total number of actual intrusions. Formula: DR = #True Positive / (#False Negative + #True Positive).
• False Positive Rate (FP) — range: [0, 1]
  • Ratio of normal connections incorrectly classified as intrusions to the total number of actual normal connections. Formula: FP = #False Positive / (#True Negative + #False Positive).

Input / output format

Input: Network traffic connection records containing categorical and quantitative features, labeled with actual class (normal, probe, dos, u2r, r2l).

Output: Predicted label for each connection instance (normal, probe, dos, u2r, or r2l).

Scoring recipe

tp = sum(1 for p, g in zip(preds, gold) if p == 'intrusion' and g == 'intrusion')
fn = sum(1 for p, g in zip(preds, gold) if p == 'normal' and g == 'intrusion')
fp = sum(1 for p, g in zip(preds, gold) if p == 'intrusion' and g == 'normal')
tn = sum(1 for p, g in zip(preds, gold) if p == 'normal' and g == 'normal')
dr = tp / (fn + tp) if (fn + tp) > 0 else 0.0
fp_rate = fp / (tn + fp) if (tn + fp) > 0 else 0.0
return dr, fp_rate

Common pitfalls

• The system ignores non-numerical features, which disproportionately hurts performance on the 'normal' class.
• Evaluation collapses the original 5-class problem into a binary Normal vs. Intrusion task for DR/FP, obscuring per-class confusion (e.g., r2l has only 5.4% accuracy).

Evidence (verbatim from paper)

Detection rate (DR) is calculated as the ratio between the number of correctly detected intrusions and the total number of intrusions [33], that is: $$ DR = \frac {# \text {T r u e P o s i t i v e}}{# \text {F a l s e N e g a t i v e} + # \text {T r u e P o s i t i v e}} $$ Using table 3, detection rate, $\mathrm{{DR}} = {0.9500}$ . False positive rate (FP) is calculated as the ratio between the numbers of normal connections that are incorrectly classified as intrusions and the total number of normal connections [33], that is: $$ \mathrm {F P} = \frac {# \text {F a l s e P o s i t i v e}}{# \text {T r u e N e g a t i v e} + # \text {F a l s e P o s i t i v e}} $$ Using table 3, false positive rate, $\mathrm{FP} = 0.3046$

Citation

@misc{hoque2012ids,
  title={An Implementation of Intrusion Detection System Using Genetic Algorithm},
  author={Hoque et al. (2012)},
  year={2012},
  note={arXiv:1204.1336}
}

• arXiv: 1204.1336