Motion Prediction Eval

motion-prediction-eval

Multimodal Interaction-aware Motion Prediction for Autonomous Street Crossing — Radwan et al. (2018) (arXiv:1808.06887, 2018)

What this evaluates

Evaluates a model's ability to predict future trajectories of pedestrians and other agents in crowded urban environments. It probes how well the architecture captures inter-agent dynamics and interaction patterns over short temporal windows to estimate safe crossing paths.

Datasets

• L-CAS — total ?; splits: test (-1)
• ETH-Hotel — total ?; splits: test (-1)
• UCY-Uni — total ?; splits: test (-1)
• ETH-Univ — total ?; splits: test (-1)
• Zara01 — total ?; splits: test (-1)
• Zara02 — total ?; splits: test (-1)

Metrics

• Average Displacement Error (ADE) (primary) — range: meters
  • Mean squared error over all predicted and ground-truth points in the trajectory.
• Final Displacement Error (FDE) — range: meters
  • Euclidean distance between the predicted and ground-truth poses at the end of the prediction interval.

Input / output format

Input: Sequence of observed agent states (spatial coordinates, velocity, orientation) over an 8-frame (3.2s) sliding observation window.

Output: Predicted future agent states (positions, orientations) over a 12-frame (4.8s) prediction horizon.

Scoring recipe

def compute_ade(pred_traj, gt_traj):
    # pred_traj, gt_traj: (T, 2) arrays of x,y coordinates
    errors = np.sqrt(np.sum((pred_traj - gt_traj)**2, axis=1))
    return np.mean(errors)

def compute_fde(pred_traj, gt_traj):
    return np.sqrt(np.sum((pred_traj[-1] - gt_traj[-1])**2))

Common pitfalls

• Baseline results are copied directly from other papers rather than re-evaluated on the same splits, potentially introducing unfair comparisons due to differing data preprocessing or splits.
• The observation window is implemented as a sliding buffer, allowing predictions immediately after a 5-second initialization phase, not after waiting for the full observation window to elapse.
• Tables often report only translational error (meters), omitting rotational error (degrees) which is also computed and reported in the text.

Evidence (verbatim from paper)

We evaluate the accuracy of our motion prediction model by reporting the following metrics: Average Displacement Error: mean squared error over all predicted and groundtruth points in the trajectory. Final Displacement Error: distance between the predicted and groundtruth poses at the end of the prediction interval.

Citation

@misc{radwan2018multimodal,
  title={Multimodal Interaction-aware Motion Prediction for Autonomous Street Crossing},
  author={Radwan et al. (2018)},
  year={2018},
  note={arXiv:1808.06887}
}

• arXiv: 1808.06887