# Moisesdb Eval

> Evaluates the ability of audio source separation models to isolate individual instrument stems from mixed stereo recordings. It probes fine-grained separation capabilities across a hierarchical taxonomy of up to 11 stems, testing robustness to stem imbalance and un-mastered audio characteristics. Use when the user wants to benchmark on MoisesDB, or asks about evaluating this task. Reports SDR.

- Skill: `qhjqhj00/moisesdb-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/moisesdb-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/moisesdb-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/moisesdb-eval

---


# moisesdb-eval

> Moisesdb: A dataset for source separation beyond 4-stems — Pereira et al. (2023) (arXiv:2307.15913, 2023)

## What this evaluates

Evaluates the ability of audio source separation models to isolate individual instrument stems from mixed stereo recordings. It probes fine-grained separation capabilities across a hierarchical taxonomy of up to 11 stems, testing robustness to stem imbalance and un-mastered audio characteristics.

## Datasets

- **MoisesDB** — total 240; splits: test (-1); repo https://github.com/moises-ai/moises-db

## Metrics

- `SDR` **(primary)** — range: dB
  - Signal-to-Distortion Ratio in decibels. Computed as 10 * log10(||s_true||^2 / ||s_true - s_est||^2), where s_true is the ground truth stem and s_est is the estimated stem. Higher values indicate better separation quality.

## Input / output format

**Input**: Stereo mixed audio recording of a song.

**Output**: One or more separated audio stems (waveform files) corresponding to the requested instrument classes (e.g., vocals, drums, bass, piano, guitar, etc.).

## Scoring recipe

```python
def compute_sdr(gt, est):
    num = np.sum(gt ** 2)
    den = np.sum((gt - est) ** 2)
    return 10 * np.log10(num / den) if den > 0 else float('inf')

def evaluate(dataset, preds):
    scores = []
    for track in dataset:
        for stem in track.stems:
            scores.append(compute_sdr(track[stem], preds[track.id][stem]))
    return np.mean(scores)
```

## Common pitfalls

- The dataset contains un-mastered mixes with lower loudness and higher dynamic range, causing distributional shift when evaluating on commercial mastered tracks.
- Stem and track frequencies follow a power-law distribution, making underrepresented stems (e.g., 'wind', 'contrabass') significantly harder to separate and potentially skewing overall averages.
- Not all songs contain all stems; the number of available tracks varies per stem configuration (e.g., N=235 for 4-stem, N=104 for 5-stem), so metrics must be averaged over available tracks rather than assuming a fixed set.

## Evidence (verbatim from paper)

> Table 3: Mean, standard deviation (Std), and median (Mdn) of the SDR in dB for each Model/Method and stem type. The varying number of available tracks is denoted by N. *Overall* indicates performance over all tracks regardless of stem group.

## Citation

```bibtex
@misc{pereira2023moisesdb,
  title={Moisesdb: A dataset for source separation beyond 4-stems},
  author={Pereira et al. (2023)},
  year={2023},
  note={arXiv:2307.15913}
}
```

- arXiv: 2307.15913