Ucla Asv Eval

ucla-asv-eval

Variable frame rate-based data augmentation to handle speaking-style variability for automatic speaker verification — Afshan et al. (2020) (arXiv:2008.03616, 2020)

What this evaluates

Evaluates automatic speaker verification (ASV) robustness to speaking-style mismatches between enrollment and test utterances. It measures how well data augmentation techniques can compensate for style variability without requiring multi-style training data.

Datasets

• UCLA database — total 240; splits: development (435), enrollment (252), test (240)

Metrics

• EER (primary) — range: percent
  • Equal Error Rate: the operating point where the False Acceptance Rate (FAR) equals the False Rejection Rate (FRR). Computed across a range of decision thresholds and reported as a percentage.

Input / output format

Input: 30-second speech utterances for enrollment and test pairs. Features (e.g., x-vectors) are extracted and processed by a PLDA classifier.

Output: A verification score or binary decision per enrollment-test pair, aggregated to compute FAR/FRR and EER across thresholds.

Scoring recipe

def compute_eer(scores, labels, thresholds):
    far, frr = [], []
    for t in thresholds:
        far.append(mean((scores[labels==1] < t)))
        frr.append(mean((scores[labels==0] > t)))
    idx = argmin(abs(array(far) - array(frr)))
    return far[idx] * 100

Common pitfalls

• Style-mismatched EERs are significantly higher than style-matched ones; readers must carefully match enrollment/test pairs in Table 2 to avoid misinterpreting baseline performance.
• VFR normalization is only applied to the development set for PLDA adaptation, not to enrollment/test utterances, to preserve speaker-specific information, which limits its direct comparability to full normalization methods.
• The multi-style configuration serves as a best-case upper bound but is noted as unrealistic for practical deployment where all speaking styles per speaker are unavailable.

Evidence (verbatim from paper)

System performance in terms of the EER for the UCLA database is shown in Table 2. Statistical significance was verified using McNemar’s test. Unless mentioned explicitly, all performance differences reported in this section are significant with p<0.005.

Citation

@misc{afshan2020variable,
  title={Variable frame rate-based data augmentation to handle speaking-style variability for automatic speaker verification},
  author={Afshan et al. (2020)},
  year={2020},
  note={arXiv:2008.03616}
}

• arXiv: 2008.03616