Hsad Eval

hsad-eval

When Fine-Tuning is Not Enough: Lessons from HSAD on Hybrid and Adversarial Audio Spoof Detection — Bin Hu et al. (arXiv:2509.07323, 2025)

What this evaluates

Evaluates audio spoof detection models on a newly constructed hybrid spoofing benchmark. It probes robustness against complex, real-world adversarial conditions including mixed-source speech, environmental noise, channel filtering, and compression artifacts.

Datasets

• Hybrid Spoofed Audio Dataset (HSAD) — total 42292; splits: test (8500)

Metrics

• Accuracy (primary) — range: percent
  • Accuracy = (C / N) × 100%, where C is the number of correct predictions and N is the total number of samples.
• F1-score — range: [0, 1]
  • Harmonic mean of precision and recall computed across genuine, AI-cloned, AI-generated, and hybrid classes.
• FPR — range: [0, 1]
  • False positive rate: proportion of genuine human samples incorrectly classified as spoofed.
• FNR — range: [0, 1]
  • False negative rate: proportion of spoofed/hybrid samples incorrectly classified as genuine.

Input / output format

Input: 16 kHz audio recordings converted to 128-bin log-Mel spectrograms (25 ms Hamming window, 10 ms frame shift). Includes clean and degraded (noisy, filtered, compressed) variants.

Output: Multi-class label (0: genuine human, 1: AI-cloned, 2: AI-generated, 3: hybrid/mixed) or reliability score.

Scoring recipe

accuracy = (correct_predictions / total_samples) * 100
fpr = false_positives / (false_positives + true_negatives)
fnr = false_negatives / (false_negatives + true_positives)
f1 = 2 * (precision * recall) / (precision + recall)
reliability_threshold = 0.5
y_hat = 0 if abs(real_tag - reliability_score) < reliability_threshold else 1

Common pitfalls

• Pretrained models frequently collapse to predicting a single dominant class (e.g., AI-generated) without fine-tuning.
• Hybrid/mixed classes cause high prediction variance and overlapping reliability scores, leading to false positives on genuine speech.
• Standard binary spoofing thresholds fail on multi-class hybrid compositions without spoof-specific calibration.

Evidence (verbatim from paper)

Performance was measured using classification accuracy, F1-score, false positive rate (FPR), and false negative rate (FNR) across the four spoofing classes. Overall classification accuracy was computed as: Accuracy = (C / N) × 100% where C is the number of correct predictions and N is the total number of samples.

Citation

@misc{hu2025hsad,
  title={When Fine-Tuning is Not Enough: Lessons from HSAD on Hybrid and Adversarial Audio Spoof Detection},
  author={Bin Hu et al.},
  year={2025},
  note={arXiv:2509.07323}
}

• arXiv: 2509.07323