# Voice Search Wer Eval

> Evaluates speech recognition accuracy and latency trade-offs for streaming vs. non-streaming decoding on a proprietary voice-search dataset. It probes the model's ability to maintain low word error rate while minimizing output delay and computational overhead. Use when the user wants to benchmark on Voice Search, or asks about evaluating this task. Reports WER.

- Skill: `qhjqhj00/voice-search-wer-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/voice-search-wer-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/voice-search-wer-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/voice-search-wer-eval

---


# voice-search-wer-eval

> Transformer Transducer: One Model Unifying Streaming and Non-streaming Speech Recognition — Tripathi et al. (2020) (arXiv:2010.03192, 2020)

## What this evaluates

Evaluates speech recognition accuracy and latency trade-offs for streaming vs. non-streaming decoding on a proprietary voice-search dataset. It probes the model's ability to maintain low word error rate while minimizing output delay and computational overhead.

## Datasets

- **Voice Search** — total ?; splits: test (14000)

## Metrics

- `WER` **(primary)** — range: percent
  - Word Error Rate: the percentage of words incorrectly recognized relative to the reference transcript.
- `RTF` — range: other
  - Real-Time Factor: the ratio of processing time to audio duration.
- `Alignment Delay` — range: other
  - Mean difference in word alignment timestamps between reference and model: D = (1/N) * sum(T_ref - T_Y).

## Input / output format

**Input**: 30ms logmel energy features (128-dim, stacked 4x, subsampled 3x) derived from speech waveforms using a 32ms window with 10ms shift.

**Output**: Transcribed text sequence and word alignment timestamps.

## Scoring recipe

```python
def compute_wer(predictions, references):
    errors = sum(edit_distance(p, r) for p, r in zip(predictions, references))
    total = sum(len(r) for r in references)
    return 100 * errors / total

def compute_alignment_delay(ref_times, model_times):
    return sum(t_ref - t_model for t_ref, t_model in zip(ref_times, model_times)) / len(ref_times)
```

## Common pitfalls

- Confusing fixed lookahead latency (e.g., 240ms buffer) with actual word alignment delay, which varies by model configuration.
- Reporting RTF without specifying hardware (TPU vs CPU) and batch size, as inference speed heavily depends on these factors.
- Overlooking that constrained alignment training improves delay but degrades low-latency WER.

## Evidence (verbatim from paper)

> Table 1 shows WER and benchmarking results for different label encoder architectures.

## Citation

```bibtex
@misc{tripathi2020transformertransducer,
  title={Transformer Transducer: One Model Unifying Streaming and Non-streaming Speech Recognition},
  author={Tripathi et al. (2020)},
  year={2020},
  note={arXiv:2010.03192}
}
```

- arXiv: 2010.03192