Hsad Spoof Detection Eval

hsad-spoof-detection-eval

Hybrid Audio Detection Using Fine-Tuned Audio Spectrogram Transformers: A Dataset-Driven Evaluation of Mixed AI-Human Speech — Huang et al. (2025) (arXiv:2505.15136, 2025)

What this evaluates

This evaluation probes the ability of audio classification models to detect and distinguish between genuine human speech, AI-cloned speech, AI-generated speech, and complex hybrid compositions that mix human and synthetic segments. It specifically tests robustness against multi-source spoofing attacks and real-world signal degradations like environmental noise, channel filtering, and codec compression.

Datasets

• ASVspoof 2019 Logical Access (LA) — total 71237; splits: train (-1), dev (-1), eval (-1)
• Proposed Hybrid Spoofed Audio Dataset (HSAD) — total ?; splits: train (-1), test (-1)

Metrics

• Accuracy (primary) — range: percent
  • Computed as (C / N) * 100%, where C is the number of correct predictions and N is the total number of samples.
• F1-score — range: [0, 1]
  • Harmonic mean of precision and recall across the four spoofing classes.
• False Positive Rate (FPR) — range: [0, 1]
  • Ratio of false positives to the total number of actual negatives.
• False Negative Rate (FNR) — range: [0, 1]
  • Ratio of false negatives to the total number of actual positives.

Input / output format

Input: 16 kHz audio recordings converted to 128-bin log-Mel spectrograms using a 25ms Hamming window and a 10ms frame shift.

Output: Multi-class classification labels (0: genuine human, 1: AI-cloned, 2: AI-generated, 3: hybrid) or continuous reliability scores for binary spoof detection.

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    total = len(gold_labels)
    return (correct / total) * 100.0

Common pitfalls

• Overlapping reliability score distributions between genuine and spoofed classes can cause models to misclassify all genuine human utterances as spoofed.
• Models fine-tuned on narrow synthetic speech generators often fail to generalize to broader or hybrid spoof distributions present in real-world benchmarks.
• Hybrid/mixed-source audio samples produce highly dispersed confidence scores due to complex boundary conditions between human and synthetic segments, leading to high variance in evaluation metrics.

Evidence (verbatim from paper)

Performance was measured using classification accuracy, F1-score, false positive rate (FPR), and false negative rate (FNR) across the four spoofing classes. ... Overall classification accuracy was computed as: $$ \mathrm{Accuracy} = \frac{C}{N}\times 100% $$ where C is the number of correct predictions and N is the total number of samples.

Citation

@misc{huang2025hybrid,
  title={Hybrid Audio Detection Using Fine-Tuned Audio Spectrogram Transformers: A Dataset-Driven Evaluation of Mixed AI-Human Speech},
  author={Huang et al. (2025)},
  year={2025},
  note={arXiv:2505.15136}
}

• arXiv: 2505.15136