Sea Spoof Eval

sea-spoof-eval

SEA-Spoof: Bridging The Gap in Multilingual Audio Deepfake Detection for South-East Asian — Jinyang Wu et al. (2025) (arXiv:2509.19865, 2025)

What this evaluates

This benchmark evaluates audio deepfake detection models on their ability to distinguish real speech from synthetic speech across six South-East Asian languages. It specifically probes cross-lingual generalization and robustness against diverse open-source and commercial text-to-speech and voice conversion systems.

Datasets

• SEA-Spoof — total 711; splits: train (439000), val (57000), test (57000)

Metrics

• EER (%) (primary) — range: percent
  • Equal Error Rate is the operating point where the False Acceptance Rate (FAR) equals the False Rejection Rate (FRR). It is computed by sweeping a decision threshold over the model's output scores and finding the intersection of the FAR and FRR curves, then expressed as a percentage.

Input / output format

Input: Raw audio clips (bona-fide or spoofed) paired with binary authenticity labels.

Output: Binary classification prediction (real vs. spoof) or a continuous spoof probability score used to compute detection thresholds.

Scoring recipe

def compute_eer(scores, labels):
    # scores: model spoof probability, labels: 1=spoof, 0=bona-fide
    far, frr, _ = compute_roc_curve(scores, labels)
    idx = np.argmin(np.abs(far - frr))
    return far[idx] * 100

Common pitfalls

• Models trained on Western benchmarks (e.g., ASVspoof) suffer severe cross-lingual performance collapse when evaluated on SEA languages without fine-tuning.
• Fine-tuning on SEA-Spoof can cause catastrophic forgetting, degrading performance on original training benchmarks like ASVspoof5.
• Commercial TTS/VC systems generate significantly harder fakes than open-source models, leading to higher EERs if not accounted for in split design.

Evidence (verbatim from paper)

Results (Table[2]) reveal severe cross-domain mismatch: MoLEx achieves strong performance on ASVspoof5 (1.25% EER) but collapses on SEA-Spoof (43.8% EER).

Citation

@misc{wu2025seaspoof,
  title={SEA-Spoof: Bridging The Gap in Multilingual Audio Deepfake Detection for South-East Asian},
  author={Jinyang Wu et al. (2025)},
  year={2025},
  note={arXiv:2509.19865}
}

• arXiv: 2509.19865