Bench2drive Personalized Driving Eval

bench2drive-personalized-driving-eval

Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving — Wang et al. (2026) (arXiv:2603.25740, 2026)

What this evaluates

This evaluation probes a vision-language-action model's ability to align autonomous driving behavior with both long-term individual driver habits and short-term natural language style instructions. It measures safety, efficiency, comfort, and stylistic fidelity in closed-loop simulation scenarios like merging, overtaking, and emergency braking.

Datasets

• Bench2Drive — total ?; splits: test (-1)

Metrics

• Driving Score (DS) (primary) — range: [0, 100]
  • Standard closed-loop driving score from Bench2Drive, typically scaled to [0, 100]. It penalizes collisions, traffic violations, and unsafe time-to-collision (TTC) while rewarding route completion.
• Alignment Score (AS) — range: [0, 1]
  • Accuracy of correctly classifying model rollouts into the target driver's historical behavior cluster after clustering all drivers using their logs.
• Ratings — range: [1, 10]
  • Human evaluator rating on a 1-10 scale measuring the perceived similarity between the driver's own logs and the model's rollouts.
• Speed — range: m/s
  • Mean average speed of the ego vehicle during the rollout.
• Acceleration — range: m/s²
  • Mean average longitudinal acceleration of the ego vehicle.
• Headway — range: m
  • Mean average following distance to the vehicle ahead.

Input / output format

Input: Multi-modal inputs including ego-vehicle camera frames, route/navigation waypoints, long-term user embedding vectors derived from historical logs, and short-term natural language style instructions (e.g., 'Aggressive', 'Conservative', 'Neutral').

Output: Continuous driving control commands (steering angle, acceleration, braking) executed at each simulation timestep to navigate the route.

Scoring recipe

def compute_metrics(rollout, gold_driver_logs):
    # Driving Score & Success Rate (Bench2Drive convention)
    ds = 100.0
    success = True
    for step in rollout:
        if step.collision or step.violation:
            ds = 0.0
            success = False
            break
        ds *= step.safety_weight
    
    # Alignment Score (AS)
    clusters = cluster_historical_logs(gold_driver_logs)
    correct = 0
    for driver in drivers:
        rollout = generate_rollout(model, driver.profile)
        predicted_cluster = classify_behavior(rollout)
        if predicted_cluster == clusters[driver.id]:
            correct += 1
    as_score = correct / len(d drivers)
    
    # Style/Performance Metrics
    speed = mean([s.velocity for s in rollout])
    accel = mean([s.long_accel for s in rollout])
    headway = mean([s.following_dist for s in rollout])
    return ds, success, as_score, speed, accel, headway

Common pitfalls

• Original Bench2Drive continues episodes after collisions; this evaluation explicitly terminates the route immediately upon collision to emphasize safety-critical assessment.
• Alignment Score (AS) relies on clustering historical logs and classifying rollouts, not direct metric matching or Euclidean distance to gold logs.
• Style instructions (short-term) and driver profiles (long-term) are evaluated separately but interactively; conflating them or ignoring the adaptive reward weighting skews personalization results.

Evidence (verbatim from paper)

We employ metrics that assess both driving performance and stylistic alignment. The Driving Score (DS), Success Rate (SR), Efficiency (Effic.), and Comfort are adopted from Bench2Drive. To quantify the personalization of the model, we measure the mean value of driving metrics, including average Speed in m/s, Acceleration (Acce.) in m/s², Lane Change Counts (LC), Headway in m, and Travel Time (TT) in s. We introduce the Alignment Score (AS) for user studies to evaluate how well the policy aligns with individual driving preferences.

Citation

@misc{wang2026drivemyway,
  title={Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving},
  author={Wang et al. (2026)},
  year={2026},
  note={arXiv:2603.25740}
}

• arXiv: 2603.25740