Cdfsl V Eval

cdfsl-v-eval

CDFSL-V: Cross-Domain Few-Shot Learning for Videos — Samarasinghe et al. (2023) (arXiv:2309.03989, 2023)

What this evaluates

Cross-domain few-shot video action recognition. It probes a model's ability to adapt to new video domains using a large source dataset and unlabeled target videos, with evaluation restricted to a 5-way 5-shot setting where only five target classes are tested with five labeled support examples each.

Datasets

• Kinetics-100 — total ?; splits: train (-1), val (-1), test (-1)
• Kinetics-400 — total ?; splits: train (-1)
• UCF101 — total ?; splits: test (-1)
• HMDB51 — total ?; splits: test (-1)
• Something-SomethingV2 — total ?; splits: test (-1)
• Diving48 — total ?; splits: test (-1)
• RareAct — total ?; splits: test (-1)

Metrics

• 5-way 5-shot accuracy (primary) — range: percent
  • Percentage of correctly classified videos in the target test split, evaluated over 5 randomly selected classes with 5 labeled support examples per class.

Input / output format

Input: Videos sampled at 16 frames at 112x112 resolution. A 5-way 5-shot support set from the target test split provides labeled examples for adaptation. Unlabeled target videos are used for consistency learning during training.

Output: Class label predictions for each video in the target test split.

Scoring recipe

def compute_accuracy(predictions, labels):
    correct = sum(1 for p, l in zip(predictions, labels) if p == l)
    return correct / len(labels) * 100

Common pitfalls

• Overlapping classes between source (Kinetics) and target datasets (UCF101, HMDB51) must be explicitly removed before training/evaluation to prevent data leakage and unfair cross-domain evaluation.
• The evaluation is strictly 5-way 5-shot, meaning only 5 target classes are tested per run with exactly 5 labeled support examples, which differs from standard full-shot or many-shot benchmarks.

Evidence (verbatim from paper)

For evaluation, we compute the 5-way 5-shot accuracy on the test-split for each target dataset. Due to class overlap between Kinetics and two of our target datasets, UCF101 and HMDB51, we remove the overlapping classes from the source dataset. Without this removal, the supervised training on shared classes between the source and target datasets would be an unfair representation of the Cross-Domain Few-Shot problem setting.

Citation

@misc{samarasinghe2023cdfslv,
  title={CDFSL-V: Cross-Domain Few-Shot Learning for Videos},
  author={Samarasinghe et al. (2023)},
  year={2023},
  note={arXiv:2309.03989}
}

• arXiv: 2309.03989