Voxceleb1 Verification Eval

voxceleb1-verification-eval

VoxCeleb2: Deep Speaker Recognition — Chung et al. (2018) (arXiv:1806.05622, 2018)

What this evaluates

Evaluates speaker verification capability by measuring how well a model distinguishes same-speaker from different-speaker audio pairs. It probes robustness across different evaluation conditions, including a standard test set, an extended large-scale set, and a constrained same-nationality/gender set.

Datasets

• VoxCeleb1 — total ?; splits: test (-1), VoxCeleb1-E (581480), VoxCeleb1-H (552536)

Metrics

• EER (%) (primary) — range: percent
  • Equal Error Rate: the operating point on the DET curve where the false acceptance rate (FAR) equals the false rejection rate (FRR).
• C^{min}_{det} — range: cost
  • Minimum Detection Cost Function: $C_{det} = C_{miss} \cdot P_{tar} \cdot F_{miss} + C_{fa} \cdot P_{non} \cdot F_{fa}$, minimized over all decision thresholds. Lower is better.

Input / output format

Input: Pairs of audio utterances (or their corresponding speaker embeddings) for binary verification.

Output: A similarity score or verification decision per pair.

Scoring recipe

# Compute FAR and FRR across thresholds
far, frr = compute_far_frr(scores, labels)
# EER is the threshold where FAR == FRR
eer = interpolate(far, frr)
# minDCF uses standard cost parameters (e.g., C_miss=1, C_fa=1, P_tar=0.01)
c_det = c_miss * p_tar * frr + c_fa * (1 - p_tar) * far
min_dcf = min(c_det)

Common pitfalls

• Overfitting to small test sets by optimizing directly on them rather than using held-out or extended sets like VoxCeleb1-E.
• Ignoring test-time augmentation methods, which significantly impact reported scores (indicated by brackets in results tables).
• Reporting only EER without minDCF, as both are required for full verification protocol compliance.

Evidence (verbatim from paper)

Table 4 provides the performance of our models on the original VoxCeleb1 test set. As might be expected, performance improves with greater network depth, and also with more training data (VoxCeleb2 vs VoxCeleb1). This also demonstrates that VoxCeleb2 provides a suitable training regime for use on other datasets.

| Models | Trained on | $C^{min}_{det}$ | EER (%) |

Citation

@misc{chung2018voxceleb2,
  title={VoxCeleb2: Deep Speaker Recognition},
  author={Chung et al. (2018)},
  year={2018},
  note={arXiv:1806.05622}
}

• arXiv: 1806.05622