Librispeech Sr Semantic Comm Eval

librispeech-sr-semantic-comm-eval

Semantic Communications for Speech Recognition — Weng et al. (2021) (arXiv:2107.11190, 2021)

What this evaluates

Evaluates the ability of semantic communication systems to transmit speech spectra over noisy wireless channels (AWGN and Rayleigh) and accurately recover text transcriptions, comparing performance against traditional speech and text transceivers.

Datasets

• LibriSpeech — total ?; splits: train (-1), test (-1)

Metrics

• Character Error Rate (CER) (primary) — range: [0, 1]
  • Standard character-level edit distance normalized by the length of the reference transcription.
• Word Error Rate (WER) — range: [0, 1]
  • Standard word-level edit distance normalized by the length of the reference transcription.

Input / output format

Input: Speech sample sequences M converted to spectra S, channel conditions H, and SNR values.

Output: Recovered text transcriptions ̂T (or ̂S).

Scoring recipe

def compute_cer_wer(predictions, references):
    cer = sum(edit_distance(p, r) for p, r in zip(predictions, references)) / sum(len(r) for r in references)
    wer = sum(edit_distance(p.split(), r.split()) for p, r in zip(predictions, references)) / sum(len(r.split()) for r in references)
    return cer, wer

Common pitfalls

• Training is performed under fixed channel conditions, but testing evaluates robustness across varying SNR regimes and channel types (AWGN vs Rayleigh).
• Benchmarks use a two-stage pipeline (source coding + ASR) rather than end-to-end semantic extraction, making direct architectural comparison non-trivial.
• The output format requires decoding CTC probabilities into text transcriptions before computing CER/WER.

Evidence (verbatim from paper)

The model is trained under fixed channel conditions and demonstrates robustness across varying channel environments, especially in low SNR regimes, outperforming traditional systems in character- and word-error rates.

Citation

@misc{weng2021semantic,
  title={Semantic Communications for Speech Recognition},
  author={Weng et al. (2021)},
  year={2021},
  note={arXiv:2107.11190}
}

• arXiv: 2107.11190