Phase Recovery Nmf Eval

phase-recovery-nmf-eval

Phase recovery in NMF for audio source separation: an insightful benchmark — Magron et al. (2016) (arXiv:1605.07469, 2016)

What this evaluates

Evaluates the effectiveness of different phase recovery and source separation algorithms on audio mixtures. It probes how well models maintain phase consistency and reconstruct audio quality under blind and oracle conditions, particularly when time-frequency bins overlap.

Datasets

• Audio source separation mixtures (synthetic harmonics, piano notes, MIDI excerpt) — total ?; splits: test (-1)

Metrics

• SDR (primary) — range: dB
  • Signal-to-Distortion Ratio in dB. Measures overall reconstruction quality by comparing the target source to the estimated source, accounting for interference, artifacts, and noise.
• SIR — range: dB
  • Signal-to-Interference Ratio in dB. Quantifies the model's ability to reject interfering sources.
• SAR — range: dB
  • Signal-to-Artifacts Ratio in dB. Measures the level of spurious artifacts introduced during separation.

Input / output format

Input: Mixed audio signals (synthetic harmonics, piano notes, or MIDI excerpts) in the time domain.

Output: Separated source signals (time-domain waveforms) with recovered phase information.

Scoring recipe

def evaluate_separation(estimates, references):
    # estimates and references are aligned 1D arrays per source
    sdr = 10 * log10(sum(references**2) / sum((references - estimates)**2))
    # SIR and SAR computed via standard BSS eval toolbox (e.g., bss_eval_sources)
    sir, sar = bss_eval_metrics(estimates, references)
    return sdr, sir, sar

Common pitfalls

• Assuming phase consistency directly correlates with audio quality; the paper shows methods like CNMF-LR enforce consistency but degrade SDR/SAR.
• Ignoring initialization sensitivity; random initialization drastically lowers SDR/SIR/SAR compared to prior NMF-based initialization.
• Confusing blind (unsupervised) and oracle (supervised) phase recovery setups, which yield significantly different performance gaps.

Evidence (verbatim from paper)

These results show that Griffin-Lim and LeRoux phase reconstruction algorithms provide poor results in terms of audio quality. While consistency is increased in NMF-GL and NMF-LR, those methods lead to a decrease of the SDR and SAR scores compared to NMF-Wiener.

Citation

@misc{magron2016phaserecovery,
  title={Phase recovery in NMF for audio source separation: an insightful benchmark},
  author={Magron et al. (2016)},
  year={2016},
  note={arXiv:1605.07469}
}

• arXiv: 1605.07469