Musdb18 Separation Eval

musdb18-separation-eval

Audio query-based music source separation — Lee et al. (2019) (arXiv:1908.06593, 2019)

What this evaluates

Probes a model's ability to isolate specific audio sources from a mixture using a provided query signal. It evaluates how well the model handles continuous latent-space conditioning and separates arbitrary or subclass instruments beyond standard training labels.

Datasets

• MUSDB18 — total 150; splits: train (100), test (50)

Metrics

• SDR (primary) — range: dB
  • Signal-to-Distortion Ratio computed using the official museval package. It measures the ratio of the target source energy to the distortion energy (artifacts, noise, interference).

Input / output format

Input: Mixture audio resampled to 22050Hz converted to a magnitude spectrogram (STFT, window 1024, 75% overlap), paired with a 3-second query audio segment encoding the target source.

Output: Separated source magnitude spectrogram, reconstructed to audio via Inverse STFT using the original mixture's phase.

Scoring recipe

import museval
import numpy as np
# Evaluate mixture against ground truth source using official package
results = museval.eval_mus(mixture_path, gt_source_path)
# Extract SDR scores for each track
sdr_scores = [res.bss['SDR'] for res in results]
# Report median SDR across the test set
median_sdr = np.median(sdr_scores)

Common pitfalls

• The model reuses the mixture's phase for reconstruction rather than predicting it, which can artificially inflate SDR compared to full-phase models.
• Performance is highly sensitive to the query latent vector; using the class mean vector yields lower SDR than using ground-truth or retrieved vectors, so baselines must be clearly specified.
• Evaluation is performed on 3-second segments, but museval aggregates metrics over full tracks; ensure consistent handling of segment boundaries.

Evidence (verbatim from paper)

We evaluated our method on the test set of MUSDB18 using the official museval package111https://sigsep.github.io/sigsep-mus-eval which computes signal-to-distortion ratio (SDR) as a quantitative measurement.

Citation

@misc{lee2019audioquery,
  title={Audio query-based music source separation},
  author={Lee et al. (2019)},
  year={2019},
  note={arXiv:1908.06593}
}

• arXiv: 1908.06593