Traffic Workzone Forecasting Eval

traffic-workzone-forecasting-eval

Accounting for Work Zone Disruptions in Traffic Flow Forecasting — Lu et al. (2024) (arXiv:2407.11407, 2024)

What this evaluates

Evaluates the ability of spatio-temporal graph neural networks to forecast traffic speed under normal and construction work zone disruption conditions. It probes how well models integrate heterogeneous work zone data to capture nonlinear spatio-temporal dependencies and maintain accuracy during significant traffic flow deviations.

Datasets

• Richmond — total ?; splits: train (-1), test_normal (-1), test_disrupted (-1)
• Tyson’s — total ?; splits: train (-1), test_normal (-1), test_disrupted (-1)

Metrics

• MAE (primary) — range: MPH
  • Mean Absolute Error: the average of the absolute differences between predicted and true traffic speeds.
• RMSE — range: MPH
  • Root Mean Squared Error: the square root of the average of squared differences between predicted and true speeds.
• MAPE — range: percent
  • Mean Absolute Percentage Error: the average of the absolute percentage differences between predicted and true speeds, calculated as |y_true - y_pred| / y_true.

Input / output format

Input: Spatio-temporal sequences of historical traffic speeds, road network topology (graph structure), and synchronized work zone event indicators for each road segment.

Output: Predicted traffic speed (in MPH) for each road segment at specified future time steps (e.g., 3, 6, or 12 steps ahead depending on sampling rate).

Scoring recipe

import numpy as np
def compute_metrics(y_true, y_pred):
    y_true, y_pred = np.asarray(y_true), np.asarray(y_pred)
    mae = np.mean(np.abs(y_true - y_pred))
    rmse = np.sqrt(np.mean((y_true - y_pred) ** 2))
    mape = np.mean(np.abs((y_true - y_pred) / y_true)) * 100
    return {'MAE': mae, 'RMSE': rmse, 'MAPE': mape}

Common pitfalls

• Missing data imputation differs between datasets (Richmond uses 0 MPH, Tyson's uses historic averages), which can artificially mask work zone impacts in Tyson's.
• Evaluation splits data into 'normal' vs 'disrupted' conditions only for testing, while training uses the full dataset. This can lead to confusion about data leakage or split methodology.
• Forecast horizons are defined in time steps that vary by dataset sampling rate (15-min vs 5-min), making direct cross-dataset metric comparisons misleading without accounting for step length.

Evidence (verbatim from paper)

Traffic speed prediction performance was quantified using three metrics: Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). The units for RMSE and MAE, which measure the error between predicted and ground-truth speed in a network segment, is miles per hour. In contrast, MAPE considers not only the error between the predicted and true speed, but also the ratio of the error to the true value.

Citation

@misc{lu2024accounting,
  title={Accounting for Work Zone Disruptions in Traffic Flow Forecasting},
  author={Lu et al. (2024)},
  year={2024},
  note={arXiv:2407.11407}
}

• arXiv: 2407.11407