Frame Auc Eval

frame-auc-eval

Multi-timescale Trajectory Prediction for Abnormal Human Activity Detection — Rodrigues et al. (2019) (arXiv:1908.04321, 2019)

What this evaluates

Evaluates a model's ability to detect abnormal human activities by predicting multi-timescale future and past pose trajectories. The framework measures prediction errors across different temporal granularities and combines them to identify anomalous frames.

Datasets

• HR-ShanghaiTech — total ?; splits: train (274515), test (42883)
• HR-Avenue — total ?; splits: train (15328), test (15324)
• Corridor — total ?; splits: train (301999), test (181567)

Metrics

• Frame-AUC (primary) — range: percent
  • Area under the Receiver Operating Characteristic (ROC) curve computed at the frame level. Binary predictions are generated by comparing combined multi-timescale pose prediction errors against a fixed threshold.

Input / output format

Input: Sequences of human pose trajectories (25 joints with confidence scores) extracted from videos, split into variable lengths (6, 10, 26, 50) corresponding to different prediction timescales.

Output: Binary anomaly label per frame (normal/abnormal) derived from voting across multi-timescale prediction errors compared to a threshold.

Scoring recipe

errors = []
for timescale in [3, 5, 13, 25]:
    pred = model.predict(pose_seq, timescale)
    err = weighted_mse(pred, gt, confidence)
    errors.append(err)
combined_error = vote(errors)
pred_label = 1 if combined_error > threshold else 0
frame_auc = compute_auc(gt_labels, pred_labels)

Common pitfalls

• The evaluation is strictly human-centric; non-human anomalies (e.g., vehicles) are ignored.
• Timescale 25 may degrade performance on datasets lacking long-term anomalies (e.g., HR-Avenue), as noted in ablation studies.
• Pose trajectories rely on external detectors and trackers, so evaluation performance is bounded by pre-processing accuracy.

Evidence (verbatim from paper)

To compare with these existing approaches, we also use Frame-AUC as the evaluating criteria. ... Finally at any time instant, if the error value is higher than a threshold, it is considered as abnormal.

Citation

@misc{rodrigues2019multi,
  title={Multi-timescale Trajectory Prediction for Abnormal Human Activity Detection},
  author={Rodrigues et al. (2019)},
  year={2019},
  note={arXiv:1908.04321}
}

• arXiv: 1908.04321