V2x Radar Eval

v2x-radar-eval

V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perception — Yang et al. (2024) (arXiv:2411.10962, 2024)

What this evaluates

Evaluates 3D object detection capabilities for autonomous driving using multi-modal sensors (LiDAR, camera, 4D radar) in single-agent (roadside and vehicle-mounted) and cooperative perception setups. It probes robustness to adverse weather conditions and communication delays in cooperative scenarios.

Datasets

• V2X-Radar — total 10000; splits: train (7000), val (1500), test (1500); repo https://github.com/yanglei18/V2X-Radar

Metrics

• AP@IoU (primary) — range: percent
  • Average Precision computed at IoU thresholds of 0.5 and 0.7 for vehicles, and 0.25 and 0.5 for pedestrians and cyclists. Results are averaged across classes and reported per difficulty level (Easy/Moderate/Hard) or distance range.

Input / output format

Input: Multi-modal sensor data (LiDAR point clouds, 4D radar point clouds, and/or RGB images) synchronized and aligned to a common coordinate system relative to the ego vehicle or roadside unit.

Output: 3D bounding boxes defined by center coordinates (x, y, z), dimensions (length, width, height), and heading angle, along with predicted class labels.

Scoring recipe

def evaluate_3d_detection(preds, gts, iou_thresh):
    tp, fp = 0, 0
    for pred in sorted(preds, key=lambda x: x.confidence, reverse=True):
        matched = False
        for gt in gts:
            if pred.class == gt.class and not gt.used:
                if compute_3d_iou(pred.box, gt.box) >= iou_thresh:
                    gt.used = True
                    tp += 1
                    matched = True
                    break
        if not matched:
            fp += 1
    precision = tp / (tp + fp)
    recall = tp / len(gts)
    return compute_ap_curve(precision, recall)

Common pitfalls

• All reported results are evaluated on the validation set, not the held-out test set.
• 4D radar-based methods only evaluate against ground truth labels within their limited field of view, whereas LiDAR and camera methods use all annotations.
• Cooperative detection benchmarks explicitly compare synchronous (ideal) vs asynchronous (100ms delay) communication settings.

Evidence (verbatim from paper)

The dataset is divided into train/val/test sets containing 7000, 1500, and 1500 frames, respectively. The selection of samples in this setup is entirely random. The cooperative 3D object detection dataset is divided into train/val/test sets with 30, 5, and 5 sequences. Notably, all experimental results are assessed using the validation set. Table 5: Cooperative 3D object detection benchmarks for vehicle category on V2X-Radar-C. Sync. (AP@IoU = 0.7 / 0.5) ↑

Citation

@misc{yang2024v2xradar,
  title={V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perception},
  author={Yang et al. (2024)},
  year={2024},
  note={arXiv:2411.10962}
}

• arXiv: 2411.10962