Few Shot Video Classification Eval

few-shot-video-classification-eval

A Closer Look at Few-Shot Video Classification: A New Baseline and Benchmark — Zhu et al. (2021) (arXiv:2110.12358, 2021)

What this evaluates

Evaluates a model's ability to classify video actions from a small number of labeled examples per class. It probes few-shot learning capabilities by measuring accuracy across thousands of randomly sampled episodes on standard video benchmarks.

Datasets

• Kinetics — total 10000; splits: train (6400), val (1200), test (2400)
• Something-Something V2 — total 71796; splits: train (67013), val (1926), test (2857)
• complete-Kinetics — total 52925; splits: train (49325), val (1200), test (2400)

Metrics

• accuracy (primary) — range: percent
  • Mean classification accuracy computed over 10,000 randomly sampled test episodes. Calculated as the number of correctly classified query videos divided by the total number of episodes.

Input / output format

Input: Support set: n classes (default 5) with k labeled video instances each. Query set: 1 video instance. Videos are preprocessed to 224x224 resolution with T=8 frames sampled.

Output: Predicted class label for the query video.

Scoring recipe

correct = 0
for _ in range(10000):
    support, query = sample_episode(n_classes=5, k_shot=k)
    pred = model.predict(query, support)
    if pred == query.label:
        correct += 1
return (correct / 10000) * 100

Common pitfalls

• Using ImageNet pre-trained weights introduces label leakage due to semantic overlap with novel action classes, violating the few-shot assumption.
• Training on the standard small Kinetics split without pre-training leads to severe overfitting; the authors recommend the complete-Kinetics benchmark for fair evaluation.
• Meta-learning methods often prioritize temporal alignment over feature representation learning, which can be outperformed by simple linear classifiers on deeper backbones.

Evidence (verbatim from paper)

In each episode, we randomly sample $n$ classes with each class containing $k$ labeled instances as our support set and 1 instance as our query set. In the testing stage, we report the mean accuracy by randomly sampling 10,000 episodes in the experiments.

Citation

@misc{zhu2021closer,
  title={A Closer Look at Few-Shot Video Classification: A New Baseline and Benchmark},
  author={Zhu et al. (2021)},
  year={2021},
  note={arXiv:2110.12358}
}

• arXiv: 2110.12358