Xd Violence Eval

xd-violence-eval

Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision — Peng Wu et al. (arXiv:2007.04687, 2020)

What this evaluates

Evaluates a model's ability to detect violent events in untrimmed audio-visual videos under weak supervision. It probes the model's capacity to fuse complementary audio and visual cues to localize violence in class-imbalanced, long-range video sequences.

Datasets

• XD-Violence — total 4754; splits: test (-1)

Metrics

• AP (primary) — range: percent
  • Average Precision computed from the frame-level precision-recall curve (PRC). It summarizes the trade-off between precision and recall across all classification thresholds, focusing on positive samples rather than being skewed by the large number of negative frames.

Input / output format

Input: Pre-extracted audio-visual features per video: visual features (I3D RGB/Flow or C3D) and audio features (VGGish log-mel spectrograms), aligned frame-by-frame or snippet-by-snippet.

Output: A continuous violence probability score per frame/snippet.

Scoring recipe

def compute_ap(predictions, ground_truth):
    # predictions: array of violence scores per frame
    # ground_truth: binary array (1 if violent, 0 otherwise)
    precisions, recalls, _ = precision_recall_curve(ground_truth, predictions)
    # AP is the area under the precision-recall curve
    ap = np.trapz(precisions, recalls)
    return ap * 100  # reported as percentage in tables

Common pitfalls

• Using AUC instead of AP, which gives overly optimistic results on this highly class-imbalanced dataset.
• Assuming unimodal (visual-only) inputs are sufficient; the paper shows audio-visual fusion significantly boosts AP.
• Confusing online detection (sliding window) with offline detection (full video context); offline consistently outperforms online by ~5% AP.

Evidence (verbatim from paper)

we utilize the frame-level precision-recall curve (PRC) and corresponding area under the curve (average precision, AP) rather than receiver operating characteristic curve (ROC) and corresponding area under the curve (AUC) since AUC usually shows an optimistic result when dealing with class-imbalanced data, and PRC and AP focus on positive samples (violence).

Citation

@misc{wu2020multimodal,
  title={Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision},
  author={Peng Wu et al.},
  year={2020},
  note={arXiv:2007.04687}
}

• arXiv: 2007.04687