Deepurban Trajectory Eval

deepurban-trajectory-eval

DeepUrban: Interaction-Aware Trajectory Prediction and Planning for Automated Driving by Aerial Imagery — Selzer & Flohr (2026) (arXiv:2601.10554, 2026)

What this evaluates

Evaluates trajectory prediction and planning capabilities in high-density urban environments with significant vehicle-to-vulnerable-road-user interactions. It measures prediction accuracy and safety compliance using displacement errors and collision scores.

Datasets

• DeepUrban — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• ADE (primary) — range: other
  • Average Displacement Error: the mean L2 distance between predicted and ground truth trajectory coordinates over all future timesteps T.
• FDE — range: other
  • Final Displacement Error: the L2 distance between predicted and ground truth trajectory coordinates at the final future timestep T.
• Best-of-N — range: other
  • The minimum ADE or FDE value selected from a set of N predicted trajectory modes.
• Collision Score — range: other
  • Evaluates safety by counting how often predicted trajectories fall below a predefined safety threshold, indicating potential collisions. Lower scores indicate better safety performance.

Input / output format

Input: Historical agent trajectories, spatiotemporal scene graph features (map layouts, lane details), and ego-vehicle reference line.

Output: Predicted future trajectory coordinates for all agents over T timesteps, and planned control actions.

Scoring recipe

def compute_ade_fde(pred_traj, gt_traj, T):
    # pred_traj, gt_traj: shape (N_agents, T, 2)
    l2_errors = np.linalg.norm(pred_traj - gt_traj, axis=2) # (N_agents, T)
    ade = np.mean(l2_errors) # Mean over agents and timesteps
    fde = np.mean(l2_errors[:, -1]) # Mean over agents at final timestep
    return ade, fde

def compute_best_of_n(pred_modes, gt_traj, T):
    # pred_modes: list of N predicted trajectories
    errors = [compute_ade_fde(m, gt_traj, T)[0] for m in pred_modes]
    return min(errors)

def compute_collision_score(pred_traj, safety_threshold):
    # Returns count/percentage of timesteps/agents where distance < threshold
    return np.mean(np.linalg.norm(pred_traj, axis=2) < safety_threshold)

Common pitfalls

• Test set is strictly held out for future online benchmarking; all reported results in this paper are computed solely on the validation set.
• Best-of-N metric requires generating multiple trajectory modes and selecting the minimum error, which can mask average performance and requires careful mode sampling.
• Collision Score relies on a specific safety threshold defined in reference [17], making direct cross-paper comparisons difficult without the original implementation details.

Evidence (verbatim from paper)

The evaluation of trajectory predictions and planning includes key metrics such as Average Displacement Error (ADE) and Final Displacement Error (FDE), which measure the mean L2 distance and the L2 distance at the endpoint between the predicted trajectories and ground truth, respectively... The results below are computed on the validation set. Test scenarios are never touched and will be used for the upcoming online benchmarking and challenges.

Citation

@misc{selzer2026deepurban,
  title={DeepUrban: Interaction-Aware Trajectory Prediction and Planning for Automated Driving by Aerial Imagery},
  author={Selzer & Flohr (2026)},
  year={2026},
  note={arXiv:2601.10554}
}

• arXiv: 2601.10554