Egotraj Bench Eval

egotraj-bench-eval

EgoTraj-Bench: Towards Robust Trajectory Prediction Under Ego-view Noisy Observations — Liu et al. (2025) (arXiv:2510.00405, 2025)

What this evaluates

Evaluates the robustness of trajectory prediction models when historical observations are corrupted by realistic ego-view perception noise (occlusions, ID switches, ego-motion drift) compared to clean bird's-eye-view ground truth.

Datasets

• EgoTraj-TBD — total 36947; splits: train (-1), val (-1), test (-1)

Metrics

• minADE@K (primary) — range: other
  • Average Euclidean distance between predicted and ground-truth trajectory points. minADE@K selects the best-matching trajectory among K model outputs.
• minFDE@K (primary) — range: other
  • Final Euclidean distance between the last predicted and ground-truth points. minFDE@K selects the best-matching trajectory among K model outputs.

Input / output format

Input: Noisy historical trajectory in BEV space (3.2s / 8 frames) and its corresponding visibility mask. Robot trajectory is included to model interaction.

Output: Future trajectory in BEV space (4.8s / 12 frames). Models generate K candidate trajectories.

Scoring recipe

def compute_min_metrics(preds, gt, K=20):
    min_ade, min_fde = float('inf'), float('inf')
    for traj in preds[:K]:
        ade = np.mean(np.linalg.norm(traj - gt, axis=1))
        fde = np.linalg.norm(traj[-1] - gt[-1])
        if ade < min_ade: min_ade = ade
        if fde < min_fde: min_fde = fde
    return min_ade, min_fde

Common pitfalls

• Models output K candidate trajectories; using standard ADE/FDE instead of minADE@K/minFDE@K unfairly penalizes multi-modal models.
• The dataset uses a chronological split (70-10-20%) to preserve temporal coherence, so random shuffling during evaluation or training will cause data leakage.
• Historical inputs contain missing frames marked by visibility masks; models must handle these via interpolation or masking, but the benchmark only scores the final future trajectory prediction.

Evidence (verbatim from paper)

Following the most commonly used ones in trajectory prediction, we adopt Average Displacement Error (ADE) and Final Displacement Error (FDE) as primary metrics, measuring average and final Euclidean distance between estimated and ground-truth trajectories. Since we evaluate multi-modal methods that generate K candidate trajectories per agent, we report minADE@K and minFDE@K, which compute ADE and FDE of the best-matching trajectory among K outputs, rewarding models for diverse yet accurate predictions.

Citation

@misc{liu2025egotrajbench,
  title={EgoTraj-Bench: Towards Robust Trajectory Prediction Under Ego-view Noisy Observations},
  author={Liu et al. (2025)},
  year={2025},
  note={arXiv:2510.00405}
}

• arXiv: 2510.00405