Nuscenes Unified Ad Eval

nuscenes-unified-ad-eval

UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs — Liu et al. (2025) (arXiv:2511.01768, 2025)

What this evaluates

This benchmark evaluates a unified autonomous driving model across six core tasks: 3D object detection, multi-object tracking, BEV map segmentation, 3D occupancy prediction, motion prediction, and trajectory planning. It probes the model's ability to process heterogeneous multi-modal sensor data (LiDAR, multi-view cameras, and temporal sequences) and generate accurate, decoupled predictions for perception, prediction, and planning domains.

Datasets

• nuScenes — total 1000; splits: train (750), val (150), test (150)

Metrics

• NDS (primary) — range: percent
  • NuScenes Detection Score, a weighted harmonic mean of detection quality metrics (mAP, attribute accuracy, etc.) normalized to a 0-100 scale.
• mAP — range: percent
  • mean Average Precision across all 23 nuScenes object classes, computed by averaging the area under the precision-recall curve for each class.
• AMOTA — range: percent
  • Association-MOTA, measures tracking accuracy by penalizing false positives, false negatives, and identity switches based on association thresholds.
• mIoU — range: percent
  • mean Intersection over Union for BEV map segmentation, averaging the IoU across all semantic map classes (drivable area, crosswalks, etc.).
• RayIoU — range: percent
  • Ray Intersection over Union for 3D occupancy prediction, computing IoU along rays from the sensor to evaluate voxel occupancy accuracy at varying distances.
• minADE — range: meters
  • minimum Average Displacement Error, the average L2 distance between predicted and ground truth trajectories for the best-performing prediction among multiple candidates.
• L2 — range: meters
  • L2 distance for planning, measuring the Euclidean distance between the predicted and ground truth future ego-vehicle trajectory.
• collision_rate — range: percent
  • collision rate for planning, the percentage of predicted trajectories that result in a collision with static or dynamic obstacles.

Input / output format

Input: Multi-modal sensor data: LiDAR point clouds voxelized into a grid (0.3m x 0.3m x 0.25m), images from 6 surrounding cameras (processed via Swin-Tiny or ResNet-50 backbones), and temporal sequences of 4 consecutive frames fed in a streaming manner.

Output: Task-specific predictions: 3D bounding boxes with class and attributes, track IDs and states, BEV semantic segmentation masks, 3D voxel occupancy grids, future agent trajectories (multiple candidates), and future ego-vehicle trajectory.

Scoring recipe

def compute_metrics(predictions, ground_truth):
    # Detection
    nds = compute_nuscenes_detection_score(predictions.bboxes, ground_truth.bboxes)
    map_score = compute_mean_ap(predictions.bboxes, ground_truth.bboxes, classes=23)
    # Tracking
    amota = compute_amota(predictions.tracks, ground_truth.tracks)
    # Map Segmentation
    miou = compute_mean_iou(predictions.map_masks, ground_truth.map_masks)
    # Occupancy
    rayiou = compute_ray_iou(predictions.occupancy, ground_truth.occupancy)
    # Motion Prediction
    minade_car = compute_min_ade(predictions.agent_trajectories, ground_truth.agent_trajectories, target='car')
    minade_ped = compute_min_ade(predictions.agent_trajectories, ground_truth.agent_trajectories, target='pedestrian')
    # Planning
    l2_dist = compute_l2_distance(predictions.ego_trajectory, ground_truth.ego_trajectory)
    col_rate = compute_collision_rate(predictions.ego_trajectory, ground_truth.obstacles)
    return {'NDS': nds, 'mAP': map_score, 'AMOTA': amota, 'mIoU': miou, 'RayIoU': rayiou, 'minADE': (minade_car, minade_ped), 'L2': l2_dist, 'collision_rate': col_rate}

Common pitfalls

• Strictly avoid using ego-status information during training and evaluation to prevent information leakage, as some baseline methods incorrectly incorporate it.
• Motion prediction and planning tasks are only evaluated when temporal inputs (multi-frame sequences) are provided; single-frame variants do not report these metrics.
• Occupancy prediction uses the Occ3D-nuScenes validation set with distance-stratified RayIoU metrics (1m, 2m, 4m), which differ from standard voxel IoU.

Evidence (verbatim from paper)

Evaluation Metrics. We adopt task-specific metrics following established protocols: mean Average Precision (mAP) and NuScenes Detection Score (NDS) for 3D object detection; AMOTA for tracking performance; mean Intersection over Union (mIoU) for BEV map segmentation*[[4]]; RayIoU for 3D occupancy prediction[[68]]; minimum Average Displacement Error (minADE) for motion prediction[[66]]; and L2 distance along with collision rate for planning evaluation[[13]]*.

Citation

@misc{liu2025unilion,
  title={UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs},
  author={Liu et al. (2025)},
  year={2025},
  note={arXiv:2511.01768}
}

• arXiv: 2511.01768