Librispeech Asv Eval

librispeech-asv-eval

An Effective Energy Mask-based Adversarial Evasion Attacks against Misclassification in Speaker Recognition Systems — Park et al. (2026) (arXiv:2601.22390, 2026)

What this evaluates

Evaluates the robustness of speaker recognition models against adversarial audio perturbations by measuring how effectively masked energy attacks disrupt speaker verification while preserving perceptual audio quality.

Datasets

• LibriSpeech — total ?; splits: clean (-1)

Metrics

• EER (%) (primary) — range: percent
  • Equal Error Rate; the operating point where False Acceptance Rate equals False Rejection Rate during speaker verification. Higher values indicate greater attack success.
• PESQ — range: other
  • Perceptual Evaluation of Speech Quality; standard ITU-T P.862 metric measuring perceived audio distortion between clean and perturbed speech.
• SNR (dB) — range: other
  • Signal-to-Noise Ratio in decibels, calculated as 10*log10(P_signal/P_noise) between clean and adversarial audio.

Input / output format

Input: 16 kHz audio utterances from LibriSpeech clean set, converted to filterbank features for speaker encoder models.

Output: Adversarial audio samples (perturbed utterances) and verification scores used to compute EER, PESQ, and SNR.

Scoring recipe

def compute_metrics(clean_audio, adv_audio, enroll_audio, test_audio, imposter_audio, model):
    pesq_val = pesq(clean_audio, adv_audio)
    snr_val = 10 * np.log10(np.mean(clean_audio**2) / np.mean((clean_audio - adv_audio)**2))
    emb_enroll = model(enroll_audio)
    emb_test = model(test_audio)
    emb_imp = model(imposter_audio)
    scores_same = cosine_similarity(emb_enroll, emb_test)
    scores_diff = cosine_similarity(emb_enroll, emb_imp)
    eer_val = compute_eer(scores_same, scores_diff)
    return pesq_val, snr_val, eer_val

Common pitfalls

• EER direction: Higher EER indicates worse speaker verification performance, which corresponds to higher attack success in this adversarial context.
• PESQ and SNR are computed per utterance and averaged across the dataset, but the paper does not specify the exact averaging method or reference implementation.
• White-box assumption: All attacks assume full access to model gradients and architecture, which limits direct comparison to black-box evasion scenarios.

Evidence (verbatim from paper)

To evaluate the speaker similarity between generated adversarial speech utterances, we conducted an automatic speaker verification (ASV) assessment using the equal error rate (EER) as the performance metric. The experimental setup included three categories: enroll, test (utterances from the same speaker as enroll but with no overlap), and imposter (utterances from different speakers). We employed PESQ to assess the level of distortion in the distorted samples for voice quality.

Citation

@misc{park2026energy,
  title={An Effective Energy Mask-based Adversarial Evasion Attacks against Misclassification in Speaker Recognition Systems},
  author={Park et al. (2026)},
  year={2026},
  note={arXiv:2601.22390}
}

• arXiv: 2601.22390