Traffic Destination Prediction Eval

traffic-destination-prediction-eval

Efficient and Interpretable Traffic Destination Prediction using Explainable Boosting Machines — Yousif et al. (2024) (arXiv:2402.03457, 2024)

What this evaluates

Evaluates the accuracy of multi-modal trajectory forecasting models in predicting the final destination of traffic agents (pedestrians and vehicles) over a future time horizon.

Datasets

• SDD — total ?; splits: test (-1)
• InD — total ?; splits: test (-1)
• Argoverse — total ?; splits: test (-1)

Metrics

• Minimum final displacement error (primary) — range: other
  • For each predicted mode, compute the Euclidean distance between the model's predicted final position and the ground-truth final position. The reported metric is the minimum distance across all predicted modes.

Input / output format

Input: Historical trajectory sequences (positions, acceleration) and contextual features for each agent to be predicted.

Output: A set of K predicted future trajectories (one per mode) with associated probabilities, plus the final displacement error computed against ground truth.

Scoring recipe

def min_fde(predictions, ground_truth, num_modes):
    errors = []
    for k in range(num_modes):
        pred_pos = predictions[k]
        gt_pos = ground_truth
        error = np.sqrt(np.sum((pred_pos - gt_pos) ** 2))
        errors.append(error)
    return min(errors)

Common pitfalls

• Failing to take the minimum over all predicted modes when reporting the error, which unfairly penalizes multi-modal models.
• Mixing up units across datasets (SDD uses pixels, while InD and Argoverse typically use meters), leading to incorrect cross-dataset comparisons.
• Ignoring the multi-modal probability assignment via log-likelihood aggregation, which affects how mode selection is evaluated.

Evidence (verbatim from paper)

After training all models, we evaluate their performance using final displacement errors displayed in table 1. The additive models show competitive results on SDD and are comparable to state-of-the-art (SoTA) on InD without road map input. However, they do not perform as well on Argoverse. One factor for that is that cars predication is a harder problem than pedestrians' predication and rely heavily on nearby elements and roadmap. Table 1: Minimum final displacement errors for 20 modes on SDD and InD, and for 6 modes on Argoverse for EBM (ours) and a subset of SoTA previous methods

Citation

@misc{yousif2024ebmtraffic,
  title={Efficient and Interpretable Traffic Destination Prediction using Explainable Boosting Machines},
  author={Yousif et al. (2024)},
  year={2024},
  note={arXiv:2402.03457}
}

• arXiv: 2402.03457