Flowxpert Mawi Eval

flowxpert-mawi-eval

FlowXpert: Context-Aware Flow Embedding for Enhanced Traffic Detection in IoT Network — Zha et al. (2025) (arXiv:2509.20861, 2025)

What this evaluates

Evaluates a network intrusion detection model's ability to classify benign versus malicious traffic flows in real-world IoT environments. It specifically probes robustness to severe class imbalance, feature sparsity mitigation via context-aware embeddings, and temporal generalization across different time periods.

Datasets

• MAWI — total ?; splits: train (-1), test (-1), generalization (-1)

Metrics

• F1-Score (primary) — range: percent
  • Harmonic mean of Precision and Recall: 2 * (Precision * Recall) / (Precision + Recall). Evaluated separately for benign and malicious classes to account for class imbalance.
• Precision — range: percent
  • Ratio of true positive predictions to all positive predictions: TP / (TP + FP).
• Recall — range: percent
  • Ratio of true positive predictions to all actual positives: TP / (TP + FN).
• Latency — range: other
  • Time taken per inference or processing cycle, measured to assess real-time deployability.
• Throughput — range: other
  • Number of flows processed per unit time, measured to assess real-time deployability.

Input / output format

Input: Context-aware flow embeddings derived from source-host-to-destination-node associations. Raw IP addresses and port numbers are explicitly excluded to prevent label leakage. Includes both encrypted (e.g., HTTPS) and unencrypted traffic flows.

Output: Binary classification label: 'Benign' or 'Malicious'.

Scoring recipe

def compute_metrics(preds, gold):
    tp = sum(1 for p, g in zip(preds, gold) if p == g == 1)
    fp = sum(1 for p, g in zip(preds, gold) if p == 1 and g == 0)
    fn = sum(1 for p, g in zip(preds, gold) if p == 0 and g == 1)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    return precision, recall, f1

Common pitfalls

• Using overall accuracy as the primary metric, which the authors explicitly avoid due to severe class imbalance between benign and malicious traffic.
• Including raw IP addresses or port numbers in the feature set, which can reveal label information and lead to cheating behaviors or overfitting.
• Evaluating on simulated datasets like NSL-KDD or CICIDS-2017 instead of real-world MAWI data, which fails to capture practical applicability and encrypted traffic challenges.

Evidence (verbatim from paper)

We use five metrics to evaluate the performance of FlowXpert, including three commonly used metrics in machine learning algorithms: Precision, Recall, and F1-Score [[35]]. Furthermore, we include two metrics that are crucial for practical deployment: Latency and Throughput. Due to the inherent class imbalance between benign and malicious traffic, a metric such as accuracy was not used. Instead, detection performance metrics were separately evaluated for benign and malicious traffic to provide a more comprehensive and fair assessment.

Citation

@misc{zha2025flowxpert,
  title={FlowXpert: Context-Aware Flow Embedding for Enhanced Traffic Detection in IoT Network},
  author={Zha et al. (2025)},
  year={2025},
  note={arXiv:2509.20861}
}

• arXiv: 2509.20861