Musdb Sdr Eval

musdb-sdr-eval

Music Source Separation in the Waveform Domain — Defossez et al. (2019) (arXiv:1911.13254, 2019)

What this evaluates

Evaluates the ability of waveform-to-waveform models to separate individual musical instruments (drums, bass, other, vocals) from a mixed audio track. It probes the model's capacity to isolate sources while minimizing contamination and artifacts, measured against ground-truth stems.

Datasets

• MusDB — total 150; splits: train (84), val (16), test (50)

Metrics

• SDR (primary) — range: dB
  • Signal-to-Distortion Ratio measuring the log ratio between the volume of the estimated source projection onto the ground truth and the volume of the residual (contamination and artifacts). Reported in dB.

Input / output format

Input: Stereo mixture waveform (sum of drums, bass, other, vocals) sampled at 44100 Hz.

Output: Four separated stereo waveforms corresponding to drums, bass, other, and vocals.

Scoring recipe

# Compute SDR per track using the official museval package
track_medians = []
for track in test_set:
    preds = model.predict(track.mixture)
    # museval.eval_sources returns SDR for each source
    track_sdrs = museval.eval_sources(track.stems, preds)['SDR']
    track_medians.append(np.median(track_sdrs))
# Aggregate as specified: median over all tracks of the median per track
final_sdr = np.median(track_medians)

Common pitfalls

• The paper explicitly uses the median of medians for aggregation, not the mean. Using mean will yield different results.
• Must use the official museval package from the SiSec Mus 2018 campaign for consistent SDR computation.
• Pitch/tempo augmentation during training significantly hurts Conv-Tasnet performance, so it should be disabled for that specific architecture.

Evidence (verbatim from paper)

Measurements of the performance of source separation models was developed by Vincent et al. for blind source separation [Vincent et al., 2006] and reused for supervised source separation in the SiSec Mus evaluation campaign [Stöter et al., 2018]. Similarly to previous work [Stoller et al., 2018, Takahashi and Mitsufuji, 2017, Takahashi et al., 2018], we focus on the SDR (Signal to Distortion Ratio) which measures the log ratio between the volume of the estimated source projection onto the ground truth, and the volume of what is left out of this projection, typically contamination by other sources or artifacts. ... As done in the SiSec Mus competition, we report the median over all tracks of the median of the metric over each track computed using the museval package.

Citation

@misc{defossez2019music,
  title={Music Source Separation in the Waveform Domain},
  author={Defossez et al. (2019)},
  year={2019},
  note={arXiv:1911.13254}
}

• arXiv: 1911.13254