Daad X Eval

daad-x-eval

Towards Safer and Understandable Driver Intention Prediction — Karuppasamy et al. (2025) (arXiv:2510.09200, 2025)

What this evaluates

Evaluates a model's ability to predict driver maneuvers and generate hierarchical, human-understandable explanations from ego-centric driving videos. It probes spatio-temporal feature disentanglement and the impact of gaze modality on interpretability.

Datasets

• DAAD-X — total ?; splits: test (-1)

Metrics

• Acc (primary) — range: [0, 1]
  • Proportion of correctly predicted action or explanation labels out of the total number of instances.
• F1 — range: [0, 1]
  • Harmonic mean of precision and recall for action prediction.
• F1(mac) — range: [0, 1]
  • Macro-averaged F1 score across all explanation classes, treating each class equally regardless of frequency.
• F1(mic) — range: [0, 1]
  • Micro-averaged F1 score across all explanation classes, calculated by aggregating contributions of all classes before computing precision and recall.

Input / output format

Input: Ego-centric driving video frames (optionally with gaze modality provided as no gaze, overlaid, or cropped circular regions). Processed through pre-trained backbones (I3D, VideoMAE, MViTv2).

Output: Predicted action label and multi-label explanation predictions (binary activation masks for each explanation class).

Scoring recipe

def compute_metrics(preds, gold):
    acc = (preds == gold).mean()
    tp = (preds & gold).sum()
    fp = (preds & ~gold).sum()
    fn = (~preds & gold).sum()
    prec = tp / (tp + fp + 1e-8)
    rec = tp / (tp + fn + 1e-8)
    f1 = 2 * prec * rec / (prec + rec + 1e-8)
    return acc, f1

Common pitfalls

• Gaze modality handling significantly impacts results; cropped gaze regions outperform overlaid or absent gaze, but excessive crop sizes degrade performance.
• Temporal cues are critical for explanation generation; frame reshuffling disproportionately hurts transformer-based models compared to CNNs, which rely more on spatial features.
• Multi-label t-SNE visualization requires computing anchor points by averaging features per explanation class, rather than applying t-SNE directly to multi-label vectors.

Evidence (verbatim from paper)

Table 2: Evaluation on DAAD-X dataset: Evaluated baselines with (wB) and without (woB) bottleneck. Here, LTM indicates Learnable Token Merging.

Citation

@misc{karuppasamy2025towards,
  title={Towards Safer and Understandable Driver Intention Prediction},
  author={Karuppasamy et al. (2025)},
  year={2025},
  note={arXiv:2510.09200}
}

• arXiv: 2510.09200