Libri2mix Noisy Eval

libri2mix-noisy-eval

Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation — Hu et al. (2023) (arXiv:2302.11131, 2023)

What this evaluates

Evaluates the ability of end-to-end speech separation models to isolate target speakers from noisy multi-speaker mixtures. It probes noise-robustness and speaker separation capability under realistic background noise conditions.

Datasets

• Libri2Mix-noisy — total ?; splits: train-360 (-1), train-100 (-1), dev (-1), test (-1); repo https://github.com/JorisCos/LibriMix
• Libri3Mix-noisy — total ?; splits: train-360 (-1), train-100 (-1), dev (-1), test (-1); repo https://github.com/JorisCos/LibriMix

Metrics

• SI-SNRi (dB) (primary) — range: dB
  • SI-SNRi = SI-SNR(estimated, clean) - SI-SNR(mixture, clean). Measures the improvement in signal-to-noise ratio over the noisy mixture input.
• SDRi (dB) — range: dB
  • SDRi = SDR(estimated, clean) - SDR(mixture, clean). Measures the improvement in signal-to-distortion ratio over the noisy mixture input.

Input / output format

Input: 8 kHz sampled noisy speech mixtures containing 2 or 3 speakers, with background noise added at a mean SNR of -2 dB.

Output: Separated 8 kHz speech waveforms for each target speaker.

Scoring recipe

def compute_si_snr_i(est, clean, mix):
    # Align estimated signal to clean signal (optimal scaling & delay)
    est_aligned = align_and_scale(est, clean)
    si_snr_est = si_snr(est_aligned, clean)
    si_snr_mix = si_snr(mix, clean)
    return si_snr_est - si_snr_mix

Common pitfalls

• SI-SNRi reports improvement over the noisy mixture, not absolute separation quality.
• Noise is added at ~-2 dB mean SNR with 3.6 dB std dev; results are not comparable to clean Libri2Mix benchmarks.
• Models must be evaluated on the test split (11 h), not the dev split, to match reported SOTA comparisons.

Evidence (verbatim from paper)

We conduct experiments on the large-scale benchmark Libri2Mix and Libri3Mix datasets (noisy version) to evaluate our proposed approach... Our best system achieves the state-of-the-art with a SI-SNR improvement (SI-SNRi) of 16.0 dB and a Signal-to-Distortion Ratio improvement (SDRi) of 16.5 dB.

Citation

@misc{hu2023unifying,
  title={Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation},
  author={Hu et al. (2023)},
  year={2023},
  note={arXiv:2302.11131}
}

• arXiv: 2302.11131