Video Action Classification Eval

video-action-classification-eval

TAda! Temporally-Adaptive Convolutions for Video Understanding — Huang et al. (2021) (arXiv:2110.06178, 2021)

What this evaluates

Evaluates a model's ability to recognize and classify human actions in video clips by predicting action categories from sampled frames. It probes temporal dynamics modeling and spatial feature extraction capabilities in video understanding tasks.

Datasets

• Kinetics-400 — total 300000; splits: train/val/test (-1)
• Something-Something-V2 — total 220000; splits: train/val/test (-1)
• Epic-Kitchens-100 — total 90000; splits: train/val/test (-1)

Metrics

• Top-1 accuracy (primary) — range: percent
  • The proportion of samples where the ground-truth action category matches the model's highest-probability prediction.
• Top-5 accuracy — range: percent
  • The proportion of samples where the ground-truth action category appears within the model's top 5 highest-probability predictions.

Input / output format

Input: Video clips sampled with 8, 16, or 32 frames using temporal jittering. Each frame is spatially cropped to 256×256 (shorter side resized to 256). During evaluation, 3 spatial crops are used with 10 or 4 clips (K400 & EK100) or 2 clips (SSV2) uniformly sampled along the temporal dimension.

Output: Action category predictions. Evaluated by comparing predicted top-1 and top-5 classes against ground-truth labels.

Scoring recipe

def compute_topk_accuracy(predictions, labels, k=1):
    correct = 0
    for pred, label in zip(predictions, labels):
        top_k = pred.argsort()[-k:][::-1]
        if label in top_k:
            correct += 1
    return correct / len(labels)

Common pitfalls

• Evaluation aggregates results over multiple temporal clips and spatial crops; failing to average predictions across all crops/clips will yield incorrect metrics.
• EK100 uses composite verb-noun action labels; models must predict the full action label, not just verbs or nouns separately, to report valid Top-1/Top-5 accuracy.
• Frame sampling strategies (8, 16, 32 frames) and clip counts vary by dataset; using inconsistent sampling during inference breaks fair comparison.

Evidence (verbatim from paper)

For evaluation, we use three spatial crops with 10 or 4 clips (K400&EK100), or 2 clips (SSV2) uniformly sampled along the temporal dimension. Each crop has the size of 256×256, which is obtained from a video with its shorter side resized to 256. Table 8: Comparison with the top approaches on Something-Something-V2... Top-1 | Top-5

Citation

@misc{huang2021tada,
  title={TAda! Temporally-Adaptive Convolutions for Video Understanding},
  author={Huang et al. (2021)},
  year={2021},
  note={arXiv:2110.06178}
}

• arXiv: 2110.06178