Traffic Classification Eval

traffic-classification-eval

FedEdge AI-TC: A Semi-supervised Traffic Classification Method based on Trusted Federated Deep Learning for Mobile Edge Computing — Wang et al. (2023) (arXiv:2308.06924, 2023)

What this evaluates

Evaluates the ability of deep learning models to classify mobile network traffic into specific application categories (e.g., video, music, games) and background services using extracted packet flow features. It specifically probes the effectiveness of semi-supervised VAE-CNN architectures and model pruning techniques for resource-constrained edge devices.

Datasets

• Private Campus Network Dataset — total 78277; splits: train (-1), test (41451)

Metrics

• Accuracy (primary) — range: [0, 1]
  • Overall classification accuracy calculated as the number of correctly predicted flows divided by the total number of flows in the test set. Reported alongside per-class Precision, Recall, and F1-score.

Input / output format

Input: Numerical features extracted from network packet flows (PCAP files) using CICFlowMeter.

Output: Categorical application label (e.g., Bilibili, QQ music, Honor of Kings, Teamfight Tactics, Game for peace, Background, IQiyi).

Scoring recipe

def compute_accuracy(predictions, labels):
    correct = sum(1 for p, l in zip(predictions, labels) if p == l)
    return correct / len(labels)

def compute_f1_macro(predictions, labels, classes):
    tp = sum(1 for p, l in zip(predictions, labels) if p == l)
    fp = sum(1 for p, l in zip(predictions, labels) if p != l)
    fn = sum(1 for p, l in zip(predictions, labels) if p != l)
    return 2 * tp / (2 * tp + fp + fn) if (2 * tp + fp + fn) > 0 else 0.0

Common pitfalls

• Training time and hardware-dependent metrics are explicitly excluded from the evaluation protocol.
• The dataset uses a semi-supervised setup where a VAE encoder is pre-trained on unlabeled flows before the CNN classifier is fine-tuned, which differs from standard supervised baselines.
• The test set size (41,451 flows) corresponds to a fixed 45% split ratio of the total collected flows, not a standard public benchmark split.

Evidence (verbatim from paper)

The following is a description of the evaluation metrics: Precision, Recall, F1, Accuracy, and AUC. ... We have set the partition ratio to 0.45. After training and testing, we obtained the respective Confusion Matrix and Classification Report for E-CNN and single CNN using this ratio. ... resulting in a total classification accuracy of 0.7422.

Citation

@misc{wang2023fededge,
  title={FedEdge AI-TC: A Semi-supervised Traffic Classification Method based on Trusted Federated Deep Learning for Mobile Edge Computing},
  author={Wang et al. (2023)},
  year={2023},
  note={arXiv:2308.06924}
}

• arXiv: 2308.06924