# Rats Noisy Speech Eval

> Evaluates the fidelity of a simulated noisy speech generator against real VHF/UHF transmitted audio, and measures the downstream robustness of automatic speech recognition (ASR) models trained on the simulated data. Use when the user wants to benchmark on RATS Channel A, or asks about evaluating this task. Reports MSSL, WER.

- Skill: `qhjqhj00/rats-noisy-speech-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/rats-noisy-speech-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/rats-noisy-speech-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/rats-noisy-speech-eval

---


# rats-noisy-speech-eval

> DENT-DDSP: Data-efficient noisy speech generator using differentiable digital signal processors for explicit distortion modelling and noise-robust speech recognition — Guo et al. (2022) (arXiv:2208.00987, 2022)

## What this evaluates

Evaluates the fidelity of a simulated noisy speech generator against real VHF/UHF transmitted audio, and measures the downstream robustness of automatic speech recognition (ASR) models trained on the simulated data.

## Datasets

- **RATS Channel A** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `MSSL` **(primary)** — range: other
  - Multi-Scale Spectral Loss comparing real and simulated noisy audio. Lower values indicate higher simulation fidelity.
- `WER` **(primary)** — range: percent
  - Word Error Rate calculated on ASR predictions versus ground truth transcripts. Lower values indicate better recognition accuracy.

## Input / output format

**Input**: Clean speech audio (1-second chunks) for distortion modeling; clean speech audio and transcribed text for ASR training/testing.

**Output**: Simulated noisy audio waveform; ASR predictions (text transcripts) for WER calculation.

## Scoring recipe

```python
# Calculate simulation fidelity
mssl = multi_scale_spectral_loss(real_noisy_audio, simulated_noisy_audio)

# Train ASR model on simulated/real data
asr_model = ConformerDualPathASR()
asr_model.train(simulated_audio, transcripts)

# Evaluate ASR on test set
predictions = asr_model.predict(test_noisy_audio)
wer = word_error_rate(predictions, test_transcripts)
```

## Common pitfalls

- Using the full 57.4h dataset for training the distortion model instead of the specified <60s parallel subset.
- Evaluating ASR on clean test data instead of the specified real noisy test set.
- Not filtering training chunks by s2t ratio (0.4 ≤ s2t < 1.0) which excludes non-speech/silence.

## Evidence (verbatim from paper)

> To evaluate the simulation fidelity, testing clean speeches are input to the trained distortion model and the MSSL[14] is calculated between the real and simulated noisy data. ... performances in terms of WER will be compared.

## Citation

```bibtex
@misc{guo2022dentddsp,
  title={DENT-DDSP: Data-efficient noisy speech generator using differentiable digital signal processors for explicit distortion modelling and noise-robust speech recognition},
  author={Guo et al. (2022)},
  year={2022},
  note={arXiv:2208.00987}
}
```

- arXiv: 2208.00987