Ciciomt2024 Ifl Eval

ciciomt2024-ifl-eval

Incremental Federated Learning for Intrusion Detection in IoT Networks under Evolving Threat Landscape — Rehman et al. (2026) (arXiv:2603.10776, 2026)

What this evaluates

Evaluates incremental federated learning models for intrusion detection in IoT networks under evolving threat distributions. Probes the model's ability to adapt to concept drift over time while mitigating catastrophic forgetting in a federated setting.

Datasets

• CICIoMT2024 — total ?; splits: train (-1), test (-1)

Metrics

• Accuracy (Acc) (primary) — range: [0, 1]
  • Micro-averaged accuracy computed as the ratio of correctly predicted samples to the total number of test samples: sum(TP_i) / sum(TP_i + FP_i + FN_i) across all classes C.

Input / output format

Input: 45 normalized network traffic features including header/flow metadata, TCP/IP flag indicators, protocol identifiers, and statistical descriptors.

Output: 6-class categorical prediction: Benign, MQTT, DoS, DDoS, Reconnaissance, or ARP_Spoofing.

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    total = len(gold_labels)
    return correct / total if total > 0 else 0.0

Common pitfalls

• The dataset uses a strict class-capping mechanism (max 10k train, 2k test per class per time period) to control imbalance, which differs from the raw dataset distribution.
• Evaluation is conducted across a temporal timeline (t0 to t6) simulating concept drift, not as a single static train/test split.
• Only micro-averaged accuracy is reported, despite precision, recall, F1, and FAR being computed internally.

Evidence (verbatim from paper)

Model performance was assessed using standard metrics derived from the confusion matrix, including Accuracy, Precision, Recall, F1-score, False Alarm Rate (FAR). Trends across Precision, Recall and F1-score were consistent with accuracy results, therefore, we report the overall accuracy (Acc) at each time period in the paper. The overall accuracy (Acc) is computed using micro-averaging as the ratio of correctly predicted samples to the total number of test samples. $$ \mathrm {A c c} = \frac {\sum_ {i = 1} ^ {C} \mathrm {T P} _ {i}}{\sum_ {i = 1} ^ {C} \left(\mathrm {T P} _ {i} + \mathrm {F P} _ {i} + \mathrm {F N} _ {i}\right)} \tag {1} $$

Citation

@misc{rehman2026incremental,
  title={Incremental Federated Learning for Intrusion Detection in IoT Networks under Evolving Threat Landscape},
  author={Rehman et al. (2026)},
  year={2026},
  note={arXiv:2603.10776}
}

• arXiv: 2603.10776