qhjqhj00 medical-asr-denoising-eval
When De-noising Hurts: A Systematic Study of Speech Enhancement Effects on Modern Medical ASR Systems — Chondhekar et al. (2025) (arXiv:2512.17562, 2025)
What this evaluates
Evaluates the robustness of modern medical ASR models to various noise conditions and assesses whether speech enhancement preprocessing improves or degrades transcription accuracy.
Datasets
- Medical ASR Recordings (unspecified) — total ?; splits: test (-1)
Metrics
semWER(primary) — range: percent- Semantic Word Error Rate, measuring the percentage of words incorrectly transcribed based on semantic equivalence rather than exact character matching.
ΔsemWER— range: percent- Change in semWER after denoising, calculated as semWER(denoised) - semWER(noisy). Positive values indicate performance degradation.
Input / output format
Input: Audio recordings of medical speech, provided in original, noisy (background, short, Gaussian at varying SNR/amplitude), or denoised states.
Output: ASR model transcriptions (text).
Scoring recipe
def compute_semwer(audio, reference):
pred = asr_model(audio)
return compute_semantic_wer(pred, reference)
def compute_delta_semwer(noisy_audio, denoised_audio, reference):
semwer_noisy = compute_semwer(noisy_audio, reference)
semwer_denoised = compute_semwer(denoised_audio, reference)
return semwer_denoised - semwer_noisy
Common pitfalls
- Assuming speech enhancement universally improves ASR performance, whereas this study shows it consistently degrades semantic accuracy across all tested models.
- Failing to distinguish between exact character WER and semantic WER (semWER), which accounts for paraphrasing and clinical terminology variations.
- Ignoring that different noise types (background, short, Gaussian) and SNR levels cause vastly different degradation magnitudes, with Gaussian noise causing catastrophic collapse for some models.
Evidence (verbatim from paper)
Figure 4 shows the change in ASR performance resulting from the denoising process. It presents the ΔsemWER (De-noised − Noisy) for all models, directly comparing the error rates for noisy audio against the same audio after enhancement.
Citation
@misc{chondhekar2025medicalasrdenoising,
title={When De-noising Hurts: A Systematic Study of Speech Enhancement Effects on Modern Medical ASR Systems},
author={Chondhekar et al. (2025)},
year={2025},
note={arXiv:2512.17562}
}
- arXiv: 2512.17562