Lrl Spoof Srr Eval

lrl-spoof-srr-eval

When Spoof Detectors Travel: Evaluation Across 66 Languages in the Low-Resource Language Spoofing Corpus — Borodin et al. (2026) (arXiv:2603.02364, 2026)

What this evaluates

Evaluates cross-lingual robustness of spoofing countermeasures by measuring spoof rejection rates on a multilingual synthetic-speech corpus at a fixed operating point calibrated on external benchmarks. Probes how language and synthesizer identity independently affect spoof detection performance.

Datasets

• Low-Resource Language Spoofing Corpus — total ?; splits: test (-1)

Metrics

• spoof rejection rate (SRR) (primary) — range: percent
  • SRR(τ) = (1/N) * Σ 1[s(x_i) < τ], where s(x) is the CM score (higher indicates bonafide), τ is a fixed threshold, and x_i are spoofed utterances. Evaluated at a transferred EER operating point τ_EER calibrated on pooled external benchmarks.

Input / output format

Input: Spoofed audio utterances from the target corpus.

Output: A continuous CM score per utterance, where higher scores indicate bonafide and lower scores indicate spoofed speech.

Scoring recipe

# 1. Calibrate threshold on external benchmarks
all_scores = concat_scores(external_benchmarks)
tau_EER = find_threshold_where_FAR_equals_FRR(all_scores)

# 2. Evaluate SRR on target corpus
N = len(target_spoof_utterances)
spoof_scores = [model.predict(x) for x in target_spoof_utterances]
SRR = sum(1 for s in spoof_scores if s < tau_EER) / N
return SRR * 100  # percentage

Common pitfalls

• The target corpus contains only spoofed speech, so standard EER cannot be computed directly on it; a threshold must be transferred from external benchmarks.
• SRR is a spoof-only metric and does not account for false rejection rates on bonafide speech, so it does not represent a full security operating point.
• Performance differences across languages may be confounded by synthesizer identity unless CM and TTS systems are explicitly fixed during controlled comparisons.

Evidence (verbatim from paper)

Let $s(x)$ denote the CM score for utterance $x$; higher scores indicate bonafide. Given a threshold $\tau$, the spoof rejection rate (SRR) on our spoof-only corpus is $\mathrm{SRR}(\tau)=\frac{1}{N}\sum_{i=1}^{N}\mathbbm{1}\big[s(x_{i})<\tau\big],$ where ${x_{i}}{i=1}^{N}$ are spoofed utterances. Since our corpus contains no bonafide speech, we cannot compute an EER directly on it; instead, we evaluate SRR at a transferred operating point $\tau{\mathrm{EER}}$ calibrated on pooled external benchmarks (Section 4.3).

Citation

@misc{borodin2026whenspoof,
  title={When Spoof Detectors Travel: Evaluation Across 66 Languages in the Low-Resource Language Spoofing Corpus},
  author={Borodin et al. (2026)},
  year={2026},
  note={arXiv:2603.02364}
}

• arXiv: 2603.02364