Bench2drive Speed Eval

bench2drive-speed-eval

Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Driving — Shao et al. (2026) (arXiv:2603.25672, 2026)

What this evaluates

Evaluates autonomous driving policies' ability to follow explicit user commands for target speed and overtake/follow behaviors in closed-loop simulations. It measures how well models track desired speeds and execute passing maneuvers while maintaining safety, comfort, and traffic compliance.

Datasets

• Bench2Drive-Speed — total 48; splits: test (48); repo https://github.com/Thinklab-SJTU/Bench2Drive-Speed

Metrics

• Speed-Adherence Score (primary) — range: percent
  • Quantifies policy fidelity to target-speed commands while jointly assessing safety, comfort, and traffic compliance over the evaluation route.
• Overtake Score — range: percent
  • Measures the success rate of executing overtake commands relative to follow commands, penalizing safety violations and route failures.

Input / output format

Input: Ego vehicle state, RGB camera inputs, target waypoint (goal), and driving commands (target speed and overtake/follow instruction) concatenated as model input.

Output: Trajectory and control actions; only the trajectory branch output is utilized for closed-loop execution.

Scoring recipe

def compute_speed_adherence_score(trajectory, target_speed, route):
    speed_error = mean_absolute_error(trajectory.speed, target_speed)
    penalty = safety_violations(trajectory) + comfort_penalty(trajectory) + traffic_penalty(trajectory)
    return max(0, 100 - (speed_error * weight + penalty))

def compute_overtake_score(trajectory, command, route):
    if command == 'overtake':
        success = check_overtake_maneuver(trajectory) and not safety_violations(trajectory)
        return 100 if success else 0
    return 0

Common pitfalls

• Overtaking commands often trigger aggressive maneuvers that increase collision risks, leading to safety violations that reduce route completion and artificially lower the overtake score.
• Virtual target-speed annotation using long extrapolation horizons introduces monotonic trend uncertainty and amplified noise, reducing speed adherence stability compared to short horizons.
• Models trained without explicit speed commands default to single-policy behaviors and cannot follow user-specified target speeds or overtake/follow instructions.

Evidence (verbatim from paper)

Table 5: Speed-Adherence Score and Overtake Score on 48 evaluation routes of Bench2Drive-Speed. Metrics are reported for All(A), Easy (E), Medium (M), and Hard (H).

Citation

@misc{shao2026bench2drivespeed,
  title={Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Driving},
  author={Shao et al. (2026)},
  year={2026},
  note={arXiv:2603.25672}
}

• arXiv: 2603.25672