Surveillance Anomaly Detection Eval

surveillance-anomaly-detection-eval

Real-world Anomaly Detection in Surveillance Videos — Sultani et al. (2018) (arXiv:1801.04264, 2018)

What this evaluates

Evaluates a model's ability to detect and temporally localize anomalous events in long, untrimmed surveillance videos using only video-level labels. It probes the model's robustness to high intra-class variation, ambiguous normal-anomalous boundaries, and varying lighting/occlusion conditions.

Datasets

• Surveillance Anomaly Dataset — total 1900; splits: train (-1), test (-1)

Metrics

• AUC (primary) — range: percent
  • Area under the frame-based Receiver Operating Characteristic (ROC) curve. Computed by plotting the true positive rate against the false positive rate at various threshold settings on frame-level anomaly scores.

Input / output format

Input: Untrimmed surveillance video frames (resized to 240x320, 30 fps), processed in 16-frame clips and averaged into 32 non-overlapping temporal segments per video.

Output: A continuous anomaly score per temporal segment (or frame), where higher values indicate a higher likelihood of containing an anomaly.

Scoring recipe

scores = model.predict_segments(video)
gt = get_frame_level_ground_truth(video)
roc = compute_roc_curve(scores, gt)
auc = integrate(roc)
return auc * 100

Common pitfalls

• Using Equal Error Rate (EER) instead of AUC, which fails to correctly measure performance when anomalies occupy only a small fraction of long videos.
• Applying standard action recognition (whole-video binary classification) instead of temporal localization, as untrimmed videos contain mostly normal content and high intra-class variation.
• Assuming segment-level annotations are available during training; the protocol is weakly supervised, relying solely on video-level normal/anomalous labels.

Evidence (verbatim from paper)

Following previous works on anomaly detection [[27]], we use frame based receiver operating characteristic (ROC) curve and corresponding area under the curve (AUC) to evaluate the performance of our method. We do not use equal error rate (EER) [[27]] as it does not measure anomaly correctly, specifically if only a small portion of a long video contains anomalous behavior.

Citation

@misc{sultani2018realworld,
  title={Real-world Anomaly Detection in Surveillance Videos},
  author={Sultani et al. (2018)},
  year={2018},
  note={arXiv:1801.04264}
}

• arXiv: 1801.04264