# TTSDS

> Evaluates text-to-speech systems by measuring distributional distance between synthetic and real speech across five factors, producing a scalar score without subjective MOS ratings.

- Skill: `qhjqhj00/ttsds` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/ttsds`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/ttsds/raw
- Safety review: PASS (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML, Model Training & Fine-tuning
- Tags: Distribution Score, Evaluation, Libritts, Ljspeech, Text To Speech, Tts, Vctk
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-08-22
- Page: https://skillmd.com/skills/qhjqhj00/ttsds

---


# TTSDS

> TTSDS -- Text-to-Speech Distribution Score — Minixhofer et al. (2024) (arXiv:2407.12707, 2024)

## What this evaluates

Measures the distributional distance between synthetic and real speech across five key factors: environment, speaker identity, prosody, intelligibility, and general speech distribution. It evaluates TTS system quality without relying on subjective Mean Opinion Scores (MOS) or simple mean-based metrics.

## Datasets

- **LibriTTS** — total ?; splits: test (-1)
- **LJSpeech** — total ?; splits: test (-1)
- **VCTK** — total ?; splits: test (-1)
- **ESC** — total ?; splits: test (-1)
- **Blizzard Challenge Training Sets (2008, 2013, 2021)** — total ?; splits: train (-1)

## Metrics

- `TTSDS` **(primary)** — range: other
  - Computes distributional distances between synthetic and reference speech using high-dimensional embeddings (e.g., HuBERT, wav2vec 2.0) and scalar features (e.g., pitch, WER, SNR). Features are averaged per factor (environment, speaker identity, prosody, intelligibility, general speech), and factor scores are averaged to produce the final TTSDS score.

## Input / output format

**Input**: Synthetic TTS audio, reference audio, and text transcripts. Conditioning uses speaker reference waveforms from LibriTTS test set paired with unrelated transcripts.

**Output**: Scalar TTSDS score and per-factor scores (environment, speaker identity, prosody, intelligibility, general speech).

## Scoring recipe

```python
# Compute per-factor distributional distances using embeddings/scalar features
factor_scores = {}
for factor in [environment, speaker, prosody, intelligibility, general]:
    distances = compute_distribution_distance(synthetic_features[factor], reference_features[factor])
    factor_scores[factor] = mean(distances)

# Average factor scores to get final metric
ttsds_score = mean(list(factor_scores.values()))
```

## Common pitfalls

- Relies on non-public TTS systems (e.g., TTS Arena) requiring dataset reproduction.
- Uses unrelated transcripts paired with reference speakers to prevent model memorization during generation.
- Distribution-based approach avoids pitfalls of mean-based metrics that mask intra-sample variability.

## Evidence (verbatim from paper)

> For all system $	imes$ feature combinations, we compute the score as described in Section 2.4. We average all features for each factor, which gives us the corresponding factor score. Averaging all factor scores in turn gives the TTSDS score.

## Citation

```bibtex
@misc{minixhofer2024ttsds,
  title={TTSDS -- Text-to-Speech Distribution Score},
  author={Minixhofer et al. (2024)},
  year={2024},
  note={arXiv:2407.12707}
}
```

- arXiv: 2407.12707