Speech Enhancement Ms Eval

speech-enhancement-ms-eval

Semi-supervised Speech Enhancement in Envelop and Details Subspaces — Sun et al. (2016) (arXiv:1609.09443, 2016)

What this evaluates

Evaluates semi-supervised speech enhancement algorithms by measuring how well they recover clean speech from noisy mixtures across varying SNRs and noise types. It probes both perceptual quality and speech intelligibility preservation.

Datasets

• IEEE Speech + Environmental/Industrial Noise Database — total ?; splits: train (150), test (50)

Metrics

• HASQI (primary) — range: other
  • Hearing-aid speech quality index; predicts speech intelligibility ratings by modeling the effect of distortions on hearing-aid processing.
• STOI — range: other
  • Short-time objective intelligibility; measures the correlation between clean and enhanced speech envelopes in short time segments.
• NCM — range: other
  • Normalized covariance metric; computes the weighted sum of transmission index values from the envelopes of probe and response signals.
• PESQ — range: other
  • Perceptual evaluation of speech quality; standard ITU-T metric for predicting subjective speech quality.
• SegSNR — range: other
  • Segmental signal-to-noise ratio; averages SNR over short time frames to assess local noise reduction.
• SDR — range: other
  • Source to distortion ratio; measures overall quality of enhanced speech relative to clean reference.
• SIR — range: other
  • Source to interference ratio; proportional to the amount of noise reduction achieved.
• SAR — range: other
  • Source to artifact ratio; inversely proportional to speech distortion or artifacts introduced by the algorithm.

Input / output format

Input: Noisy speech signals down-sampled to 16 kHz, processed with 512-sample frames and 0.5-overlapped Hann windows to form spectrograms.

Output: Enhanced speech signals reconstructed via overlap-and-add procedure.

Scoring recipe

def compute_metrics(clean_ref, enhanced_pred):
    pesq = calculate_pesq(16000, clean_ref, enhanced_pred)
    segsnr = calculate_segsnr(clean_ref, enhanced_pred)
    sdr, sir, sar = bss_eval_sources(clean_ref, enhanced_pred)
    hasqi = calculate_hasqi(clean_ref, enhanced_pred)
    ncm = calculate_ncm(clean_ref, enhanced_pred)
    stoi = calculate_stoi(clean_ref, enhanced_pred)
    return {
        'PESQ': pesq, 'SegSNR': segsnr,
        'SDR': sdr, 'SIR': sir, 'SAR': sar,
        'HASQI': hasqi, 'NCM': ncm, 'STOI': stoi
    }

Common pitfalls

• Dictionary training set (150 speech samples) must be strictly excluded from test utterances to prevent data leakage.
• HASQI, NCM, and STOI require exact 16 kHz sampling and precise time-alignment with reference signals; mismatched rates or delays invalidate results.
• Evaluations compare modulation subspace (MS) vs. complete spectrum (NonMS) decompositions; results must be stratified by SNR and noise coherence ratio to avoid unfair conclusions.

Evidence (verbatim from paper)

To evaluate the performance of proposed algorithms on intelligibility of enhanced speech, three popular indexes, including hearing-aid speech quality index (HASQI), normalized covariance metric (NCM), and short-time objective intelligibility (STOI), are employed in this section. HASQI has great potential to specifically capture quality when speech is subjected to a wide variety of distortions. This index can accurately predict the speech intelligibility ratings and generally as an improved version of Coherence speech intelligibility index (CSII).

Citation

@misc{sun2016semi,
  title={Semi-supervised Speech Enhancement in Envelop and Details Subspaces},
  author={Sun et al. (2016)},
  year={2016},
  note={arXiv:1609.09443}
}

• arXiv: 1609.09443