Eeg Asr Noisy Speech Eval

eeg-asr-noisy-speech-eval

Advancing Speech Recognition With No Speech Or With Noisy Speech — Krishna et al. (2019) (arXiv:1906.08871, 2019)

What this evaluates

Evaluates end-to-end continuous speech recognition models using only electroencephalography (EEG) signals, and assesses robustness to background noise by fusing EEG with acoustic features.

Datasets

• Database A — total ?; splits: train (-1), val (-1), test (-1)
• Database B — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• Word Error Rate (WER) (primary) — range: percent
  • Standard character/word-level edit distance between predicted transcript and ground truth, normalized by reference length. Not explicitly defined in this section but implied by ASR context.

Input / output format

Input: 32-channel EEG time-series signals recorded simultaneously with speech utterances (and optionally acoustic audio for noisy speech fusion).

Output: Continuous text transcript of the spoken sentence.

Scoring recipe

def wer(predictions, references):
    total_errors = 0
    total_words = 0
    for pred, ref in zip(predictions, references):
        pred_words = pred.split()
        ref_words = ref.split()
        total_words += len(ref_words)
        total_errors += edit_distance(pred_words, ref_words)
    return 1 - (total_errors / total_words) if total_words > 0 else 1.0

Common pitfalls

• Subject-dependent splits: train/val/test are partitioned by participant, not by sentence, requiring careful handling to prevent data leakage across subjects.
• Noise conditions are dataset-specific (40 dB AC fan vs 65 dB music), so models trained on one cannot be directly evaluated on the other without adaptation.
• Only 30 unique sentences are used per dataset, limiting lexical coverage and potentially inflating performance on repeated utterances.

Evidence (verbatim from paper)

We built two types of simultaneous speech EEG recording databases for this work. For database A five female and five male subjects took part in the experiment. For database B five male and three female subjects took part in the experiment. Except two subjects, rest all were native English speakers for both the databases. All subjects were UT Austin undergraduate, graduate students in their early twenties. For data set A, the 10 subjects were asked to speak the first 30 sentences from the USC-TIMIT database [17] and their simultaneous speech and EEG signals were recorded. This data was recorded in presence of background noise of 40 dB (noise generated by room air conditioner fan). We then asked each subject to repeat the same experiment two more times, thus we had 30 speech EEG recording examples for each sentence. For data set B, the 8 subjects were asked to repeat the same previous experiment but this time we used background music played from our lab computer to generate a background noise of 65 dB. Here we had 24 speech EEG recording examples for each sentence. For data set A, we used data from first 8 subjects for training the model, remaining two subjects data for validation an

Citation

@misc{krishna2019advancing,
  title={Advancing Speech Recognition With No Speech Or With Noisy Speech},
  author={Krishna et al. (2019)},
  year={2019},
  note={arXiv:1906.08871}
}

• arXiv: 1906.08871