Few Shot Audio Classification Eval

few-shot-audio-classification-eval

Prototypical Contrastive Learning For Improved Few-Shot Audio Classification — Sgouropoulos et al. (2025) (arXiv:2509.10074, 2025)

What this evaluates

This benchmark evaluates few-shot audio classification capability across diverse acoustic domains including environmental sounds, musical instruments, bird species, and speaker recognition. It measures how well a model generalizes to novel classes with only a few labeled examples per class using a prototypical network framework.

Datasets

• ESC-50 — total 2000; splits: test (-1), val (-1)
• FSD2018 — total 11000; splits: test (-1), val (-1)
• NSynth — total 300000; splits: test (-1), val (-1)
• BirdCLEF 2020 — total 80000; splits: test (-1), val (-1)
• VoxCeleb1 — total 57737; splits: test (-1), val (-1)

Metrics

• accuracy (primary) — range: percent
  • Average classification accuracy computed over 2,000 randomly sampled 5-way, 5-shot tasks from the test set. Results are averaged across 5 independent experimental runs, with 95% confidence intervals reported.

Input / output format

Input: 16 kHz audio clips converted to mel spectrograms, segmented to 5 seconds if variable-length, and globally standardized. Each few-shot episode consists of a support set (5 classes × 5 samples) and a query set. Spectrograms are processed through a CRNN backbone with a self-attention mechanism to produce 256-dimensional embeddings.

Output: Class labels for query samples, determined by nearest-prototype classification in the embedding space.

Scoring recipe

episode_accuracies = []
for _ in range(2000):
    support, query = sample_episode(test_set, n_way=5, n_shot=5)
    prototypes = mean_embeddings(support)
    preds = nearest_prototype(query, prototypes)
    episode_accuracies.append(mean(preds == query.labels))
return mean(episode_accuracies)

Common pitfalls

• The evaluation relies on 2,000 randomized few-shot episodes per dataset, so results are highly sensitive to the random seed and task sampling strategy.
• Hyperparameters for the angular loss (angle threshold alpha and anchor strategy) are tuned separately per dataset rather than using a single fixed setting.
• Variable-length audio is truncated or padded to 5-second segments, which may discard contextual information or introduce artifacts not present in the original recordings.

Evidence (verbatim from paper)

We evaluate the best performing model on the validation set over 2,000 randomly sampled 5-way, 5-shot tasks from the test set. We repeat each experiment five times and report the average accuracy and the 95% confidence interval.

Citation

@misc{sgouropoulos2025prototypical,
  title={Prototypical Contrastive Learning For Improved Few-Shot Audio Classification},
  author={Sgouropoulos et al. (2025)},
  year={2025},
  note={arXiv:2509.10074}
}

• arXiv: 2509.10074