Mtass Sdr Eval

mtass-sdr-eval

Multi-Task Audio Source Separation — Lu Zhang et al. (2021) (arXiv:2107.06467, 2021)

What this evaluates

Evaluates a model's ability to simultaneously separate speech, music, and background noise from monaural audio mixtures. It measures separation fidelity using signal-to-distortion ratio and its improvement over baselines across all three source tracks.

Datasets

• Unspecified (speech, music, noise tracks) — total ?; splits: test (-1)

Metrics

• SDR (dB) — range: dB
  • Signal-to-Distortion Ratio: $10\log_{10}(|s_{\text{target}}|^2 / |e_{\text{interf}}+e_{\text{noise}}+e_{\text{artif}}|^2)$, where $s_{\text{target}}$ is the target source and error terms represent interference, noise, and artifacts.
• SDRi (dB) (primary) — range: dB
  • SDR improvement, calculated as the difference between the SDR of the separated signal and the SDR of the original mixture. Reported as the headline metric across all comparison tables.

Input / output format

Input: Monaural audio mixture containing overlapping speech, music, and background noise.

Output: Three separated audio waveforms corresponding to the speech, music, and noise tracks.

Scoring recipe

def compute_sdri(separated, mixture, target):
    sdr_sep = 10 * log10(norm(target)**2 / norm(separated - target)**2)
    sdr_mix = 10 * log10(norm(mixture)**2 / norm(mixture - target)**2)
    return sdr_sep - sdr_mix

# Compute SDRi for each track (speech, music, noise)
# Report mean SDRi across tracks as the primary evaluation score.

Common pitfalls

• Confusing absolute SDR with SDRi (improvement over mixture), leading to incorrect baseline comparisons.
• Reporting results for only one source track instead of providing balanced metrics across speech, music, and noise.
• Ignoring spectral phase when evaluating frequency-domain baselines, which unfairly penalizes complex-domain models that explicitly model phase.

Evidence (verbatim from paper)

Separation performance is measured by the signal-to-distortion ratio (SDR) [[38]], which is defined as: SDR := 10log10(||s_target||^2 / ||e_interf + e_mise + e_artif||^2) where e_interf, e_noise, and e_artif are, respectively, the interferences, noise, and artifacts error terms. Table 3: SDR improvement (SDRi) for different training targets of the proposed separator obtained on the speech, music, and noise signal tracks.

Citation

@misc{zhang2021multi,
  title={Multi-Task Audio Source Separation},
  author={Lu Zhang et al. (2021)},
  year={2021},
  note={arXiv:2107.06467}
}

• arXiv: 2107.06467