Retrotrucks Eval

retrotrucks-eval

Towards Anomaly Detection in Dashcam Videos — Haesh et al. (2020) (arXiv:2004.05261, 2020)

What this evaluates

Evaluates video anomaly detection models on dashcam footage, specifically testing their ability to detect complex traffic anomalies like collisions and skidding in dynamic, real-world driving scenes. It also benchmarks performance against standard pedestrian anomaly detection datasets to highlight challenges posed by moving cameras and contextual anomalies.

Datasets

• RetroTrucks — total ?; splits: train (-1), test (-1); repo https://drive.google.com/open?id=1VxFG1jHBiep4R3i_MmvMfKWH11AEFFhu
• UCSD Ped1 — total ?; splits: train (-1), test (-1)
• UCSD Ped2 — total ?; splits: train (-1), test (-1)
• ShanghaiTech — total ?; splits: train (-1), test (-1)

Metrics

• AUC-ROC (primary) — range: [0, 1]
  • Frame-wise Area Under the Receiver Operating Characteristic curve. Raw anomaly scores are normalized per video using min-max scaling before computing the metric against ground-truth frame labels.

Input / output format

Input: 32-frame video clips of dimensions 32 × 224 × 224 × 3 (RGB frames).

Output: Normalized frame-wise anomaly scores in [0, 1], computed via sliding-window inference.

Scoring recipe

def compute_auc_roc(raw_scores, labels):
    # Normalize scores per video
    min_s, max_s = min(raw_scores), max(raw_scores)
    norm_scores = [(s - min_s) / (max_s - min_s) for s in raw_scores]
    # Compute frame-wise AUC-ROC
    return auc_roc(norm_scores, labels)

Common pitfalls

• Anomaly scores must be normalized per video using min-max scaling before computing AUC-ROC; skipping this step skews results across videos of different scales or lengths.
• The evaluation uses a sliding-window of 32 frames with 16-frame overlap, meaning scores are assigned per frame rather than per clip, and the window must be applied consistently across all videos.

Evidence (verbatim from paper)

We then use the frame-wise AUC-ROC metric as an evaluation criterion. We then normalize the anomaly scores for each video as below: s_i = (a_i - min_i a_i) / (max_i a_i - min_i a_i), where a_i is the anomaly score of the i^{th} frame.

Citation

@misc{haesh2020towards,
  title={Towards Anomaly Detection in Dashcam Videos},
  author={Haesh et al. (2020)},
  year={2020},
  note={arXiv:2004.05261}
}

• arXiv: 2004.05261