Voice Conversion Eval

voice-conversion-eval

Non-Parallel Sequence-to-Sequence Voice Conversion with Disentangled Linguistic and Speaker Representations — Zhang et al. (2019) (arXiv:1906.10508, 2019)

What this evaluates

Evaluates non-parallel voice conversion quality by measuring spectral distortion, pitch accuracy, voicing correctness, duration modification capability, and subjective naturalness/speaker similarity.

Datasets

• VCTK — total ?; splits: test (-1)
• CMU ARCTIC — total ?; splits: test (-1)

Metrics

• MCD (primary) — range: other
  • Mel-cepstrum distortion measures the spectral distance between converted and reference utterances. Lower values indicate better spectral fidelity.
• F0 RMSE — range: other
  • Root mean square error of the fundamental frequency (pitch) between converted and reference utterances. Lower is better.
• VUV — range: percent
  • Error rate of voicing/unvoicing classification flags between converted and reference. Lower is better.
• F0 CORR — range: [0, 1]
  • Pearson correlation coefficient of the fundamental frequency trajectories between converted and reference. Higher is better.
• DDUR — range: other
  • Average absolute difference in duration between converted and reference utterances after removing boundary silence. Lower is better.
• MOS — range: [1, 5]
  • Mean Opinion Score on a 5-point scale (1: bad to 5: excellent) evaluating naturalness and similarity. Higher is better.

Input / output format

Input: Source speaker acoustic features (e.g., Mel-spectrograms or MCCs/BAPs/F0) and target speaker embedding; optionally phoneme transcriptions for linguistic supervision.

Output: Converted acoustic features or waveform matching the target speaker's voice.

Scoring recipe

# Extract F0 and MCCs from converted waveform using STRAIGHT
conv_f0, conv_mccs = straight_extract(converted_waveform)
ref_f0, ref_mccs = straight_extract(reference_waveform)
# Align converted to reference using DTW on MCCs
aligned_conv_f0, aligned_conv_mccs = dtw_align(conv_f0, conv_mccs, ref_f0, ref_mccs)
# Remove silence at boundaries
aligned_conv_f0, aligned_conv_mccs = trim_silence(aligned_conv_f0, aligned_conv_mccs)
# Compute metrics
mcd = compute_mcd(aligned_conv_mccs, ref_mccs)
f0_rmse = sqrt(mean((aligned_conv_f0 - ref_f0)**2))
vuv = error_rate(voicing_flags(aligned_conv_f0), voicing_flags(ref_f0))
f0_corr = pearson_corr(aligned_conv_f0, ref_f0)
ddur = mean(abs(duration(aligned_conv_mccs) - duration(ref_mccs)))

Common pitfalls

• F0 and MCCs must be extracted from the converted waveform using STRAIGHT, not directly from converted Mel-spectrograms, as direct extraction is non-trivial.
• Metrics require dynamic time warping (DTW) alignment of converted and reference utterances using MCCs features before computation.
• Silence segments at the beginning and end of utterances must be removed before calculating DDUR.

Evidence (verbatim from paper)

Mel-cepstrum distortion (MCD), root of mean square errors of F0 (F0 RMSE), the error rate of voicing/unvoicing flags (VUV) and the Pearson correlation factor of F0 (F0 CORR) were used as the metrics for objective evaluation. ... Because Mel-spectrograms were adopted as acoustic features in the Seq2seqVC method and our proposed method, it's not straightforward to extract F0 and MCCs features from the converted acoustic features. Therefore, the MCCs and F0 were extracted from the waveform of converted utterances using STRAIGHT [59]. Then, they were aligned to the reference utterances by dynamic time wrapping using MCCs features for calculating the metrics.

Citation

@misc{zhang2019nonparallel,
  title={Non-Parallel Sequence-to-Sequence Voice Conversion with Disentangled Linguistic and Speaker Representations},
  author={Zhang et al. (2019)},
  year={2019},
  note={arXiv:1906.10508}
}

• arXiv: 1906.10508