V2v Got Planning Eval

v2v-got-planning-eval

V2V-GoT: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models and Graph-of-Thoughts — Chiu et al. (2025) (arXiv:2509.18053, 2025)

What this evaluates

Evaluates cooperative autonomous driving planning capabilities using a multimodal LLM with graph-of-thoughts reasoning. It measures trajectory prediction accuracy and collision avoidance under occlusion-aware perception and planning-aware prediction scenarios.

Datasets

• V2V-GoT-QA — total ?; splits: test (-1)

Metrics

• L2 error (primary) — range: other
  • Mean Euclidean distance between predicted and ground-truth trajectory points over the full planning horizon.
• collision rate — range: percent
  • Percentage of test samples or simulation steps where a collision occurs between vehicles.

Input / output format

Input: Multimodal perception features from current and previous timesteps, shared planning data, and structured query-answer prompts within a graph-of-thoughts framework.

Output: Final planning answer (trajectory) generated by the MLLM after graph-of-thoughts reasoning.

Scoring recipe

def compute_metrics(predictions, ground_truth):
    l2_errors = []
    collisions = 0
    for pred, gt in zip(predictions, ground_truth):
        l2_errors.append(np.mean(np.linalg.norm(pred - gt, axis=1)))
        if check_collision(pred):
            collisions += 1
    l2_error = np.mean(l2_errors)
    collision_rate = (collisions / len(predictions)) * 100
    return l2_error, collision_rate

Common pitfalls

• L2 error is averaged over the entire planning horizon, not just the immediate next step.
• Collision rate is reported as a percentage (CR(%)), requiring careful normalization against total test samples.
• The evaluation assumes shared perception features across timesteps, so temporal alignment of inputs is critical.

Evidence (verbatim from paper)

Table I shows the testing performance of V2V-GoT in the planning task of V2V-GoT-QA in comparison with baseline methods. Our newly proposed V2V-GoT is seen to achieve the best final planning performance with the lowest L2 errors and collision rates compared to all baselines with different fusion approaches.

Citation

@misc{chiu2025v2vgot,
  title={V2V-GoT: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models and Graph-of-Thoughts},
  author={Chiu et al. (2025)},
  year={2025},
  note={arXiv:2509.18053}
}

• arXiv: 2509.18053