Low Resource Asr Eval

low-resource-asr-eval

Frustratingly Easy Data Augmentation for Low-Resource ASR — Ibaraki et al. (2025) (arXiv:2509.15373, 2025)

What this evaluates

Evaluates automatic speech recognition (ASR) performance on low-resource and high-resource languages using synthetic audio generated from text augmentation. It probes the model's ability to generalize to unseen lexical and syntactic variations when trained on limited real speech data.

Datasets

• Vatlongos — total ?; splits: train (-1)
• Nashta — total ?; splits: train (-1)
• Kakabe — total ?; splits: train (-1)
• Shinekhen Buryat — total ?; splits: train (-1)
• LibriSpeech — total ?; splits: test-clean-54 (-1), test-clean-108 (-1), test-clean-324 (-1), train-clean-1207 (-1)

Metrics

• WER (primary) — range: percent
  • Word Error Rate, calculated as the minimum number of insertions, deletions, and substitutions of words required to transform the predicted transcription into the reference transcription, divided by the total number of words in the reference.

Input / output format

Input: Raw audio waveform (synthetic or original) fed into a Wav2Vec2-XLSR-53 encoder.

Output: CTC token sequence decoded into a text transcription.

Scoring recipe

def calculate_wer(predictions, references):
    total_errors = 0
    total_words = 0
    for pred, ref in zip(predictions, references):
        dist = edit_distance(pred.split(), ref.split())
        total_errors += dist
        total_words += len(ref.split())
    return (total_errors / total_words) * 100 if total_words > 0 else 0.0

Common pitfalls

• Using a higher ratio of synthetic to original data than 1:1 degrades performance.
• Assuming semantic coherence is required; the augmentation deliberately uses hallucinated or ungrammatical text to test robustness.
• Voice cloning is unnecessary and was found to yield no performance improvements over standardized TTS voices.

Evidence (verbatim from paper)

achieving up to a 14.3% WER reduction in Nashta ASR and demonstrating broad applicability across low-resource and high-resource languages by enhancing model robustness through lexical and syntactic variation rather than semantic fidelity.

Citation

@misc{ibaraki2025frustratingly,
  title={Frustratingly Easy Data Augmentation for Low-Resource ASR},
  author={Ibaraki et al. (2025)},
  year={2025},
  note={arXiv:2509.15373}
}

• arXiv: 2509.15373