Kdd Cup Eval

kdd-cup-eval

Investigating Cellular Automata Based Network Intrusion Detection System For Fixed Networks (NIDWCA) — P.Kiran Sree and I. Ramesh Babu (2014) (arXiv:1401.3046, 2014)

What this evaluates

Evaluates the capability of an unsupervised cellular automata-based framework to detect network intrusions and anomalous traffic patterns. It probes the model's ability to learn spatial-temporal dependencies from raw network connection records and distinguish between normal and malicious activities.

Datasets

• KDD Cup — total ?; splits: train (-1), test (-1)

Metrics

• Intrusion Detection Vs Positive Rate (primary) — range: [0, 1]
  • Plots the True Positive Rate (Intrusion Detection) against the False Positive Rate (Positive Rate) across classification thresholds. Detection Rate = TP / (TP + FN); Positive Rate = FP / (FP + TN).

Input / output format

Input: Network traffic records represented as 41-dimensional feature vectors extracted from system call audit files.

Output: Binary classification labels (intrusion vs. normal) per record, aggregated into detection and positive rates for plotting.

Scoring recipe

def compute_ids_metrics(predictions, labels):
    tp = sum(1 for p, l in zip(predictions, labels) if p == 1 and l == 1)
    fp = sum(1 for p, l in zip(predictions, labels) if p == 1 and l == 0)
    fn = sum(1 for p, l in zip(predictions, labels) if p == 0 and l == 1)
    tn = sum(1 for p, l in zip(predictions, labels) if p == 0 and l == 0)
    detection_rate = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    positive_rate = fp / (fp + tn) if (fp + tn) > 0 else 0.0
    return detection_rate, positive_rate

Common pitfalls

• The paper tests the trained algorithm on the same dataset used for training, risking data leakage and overoptimistic performance estimates.
• KDD Cup lacks standardized train/test splits, making cross-study comparisons difficult without explicit partitioning details.
• Reporting only aggregate detection/positive rates obscures performance on specific attack categories (e.g., DoS vs. R2L).

Evidence (verbatim from paper)

We have applied our algorithm to a set of standard benchmark data, namely the KDD Cup data. Network traffic can be captured using packet-capturing utilities or operating system utilities at the system call level. It is stored as a collection of records in the audit file. Each data point is described by 41 features. The algorithm was trained with the KDD Cup data set. Figure 3, 4, 5 depicts the findings. Intrusion Detection Vs Positive Rate in Sample Network for Classifier I

Citation

@misc{kiran2014nidwca,
  title={Investigating Cellular Automata Based Network Intrusion Detection System For Fixed Networks (NIDWCA)},
  author={P.Kiran Sree and I. Ramesh Babu (2014)},
  year={2014},
  note={arXiv:1401.3046}
}

• arXiv: 1401.3046