Ubb Ntp Eval

ubb-ntp-eval

Analytic Network Traffic Prediction Based on User Behavior Modeling — Wang et al. (2023) (arXiv:2304.09811, 2023)

What this evaluates

Evaluates the predictive accuracy and computational efficiency of a user-behavior-based network traffic forecasting method against statistical and neural network baselines on real-world SMS traffic data.

Datasets

• Guangzhou and Milan SMS datasets — total ?; splits: train (first 2 weeks) (-1), test (rest) (-1)

Metrics

• MSE — range: other
  • Mean Square Error: the average of the squares of the errors between predicted and actual values.
• RMSE — range: other
  • Root Mean Square Error: the square root of the MSE, providing error magnitude in the original units.
• MAE — range: other
  • Mean Absolute Error: the average of the absolute differences between predicted and actual values.
• R2 (primary) — range: [0, 1]
  • Coefficient of determination: 1 minus the ratio of the residual sum of squares to the total sum of squares, indicating the proportion of variance explained by the model.

Input / output format

Input: Time-series network traffic data (SMS message counts) over a multi-week period, used to fit nine normal-distribution parameters representing user behavior cycles.

Output: Predicted traffic values for the test period, fitted behavioral parameters (peak, variance, time), and total elapsed time for training and prediction.

Scoring recipe

def compute_metrics(y_true, y_pred):
    n = len(y_true)
    y_mean = sum(y_true) / n
    mse = sum((y - yp)**2 for y, yp in zip(y_true, y_pred)) / n
    rmse = mse**0.5
    mae = sum(abs(y - yp) for y, yp in zip(y_true, y_pred)) / n
    ss_res = sum((y - yp)**2 for y, yp in zip(y_true, y_pred))
    ss_tot = sum((y - y_mean)**2 for y in y_true)
    r2 = 1 - (ss_res / ss_tot)
    return {'MSE': mse, 'RMSE': rmse, 'MAE': mae, 'R2': r2}

Common pitfalls

• The train/test split is strictly time-based (first two weeks vs. remaining weeks), not random, which may limit generalizability to different time horizons.
• Computational efficiency metrics combine training and prediction time, but the proposed method requires no iterative offline training, making the comparison with LSTM/ARIMA slightly asymmetric.
• Metrics are reported on two geographically distinct datasets with different traffic magnitudes, so absolute error values (MSE/MAE) are not directly comparable across cities without normalization.

Evidence (verbatim from paper)

Predictive accuracy is measured by Mean Square Error (MSE), Root MSE (RMSE), Mean Absolute Error (MAE), and coefficient of determination (R2), which can be clearly defined by the following formulas: ... We use the SMS data of the first two weeks to build the proposed mathematical model and the rest to evaluate the performance.

Citation

@misc{wang2023analytic,
  title={Analytic Network Traffic Prediction Based on User Behavior Modeling},
  author={Wang et al. (2023)},
  year={2023},
  note={arXiv:2304.09811}
}

• arXiv: 2304.09811