Lort Speech Enhancement Eval

lort-speech-enhancement-eval

LORT: Locally Refined Convolution and Taylor Transformer for Monaural Speech Enhancement — Wang et al. (2025) (arXiv:2509.23832, 2025)

What this evaluates

Evaluates monaural speech enhancement models by measuring how effectively they restore clean speech from noisy recordings. The benchmark probes the model's ability to handle diverse acoustic conditions and varying signal-to-noise ratios across two standard speech enhancement datasets.

Datasets

• VCTK+DEMAND — total 12396; splits: train (-1), test (-1)
• DNS Challenge 2020 — total 60000; splits: train (50000), val (5000), test (5000)

Metrics

• PESQ (primary) — range: [-0.5, 4.5]
  • ITU-T P.862 standard for perceptual speech quality. Compares enhanced speech to a clean reference signal on a scale from -0.5 to 4.5.
• STOI — range: [0, 1]
  • Short-Time Objective Intelligibility. Measures the proportion of intelligible content by analyzing time-frequency correlations between enhanced and clean speech.
• MOS (CSIG, CBAK, COVL) — range: [1, 5]
  • Mean Opinion Score predictions for signal distortion (CSIG), background noise interference (CBAK), and overall quality (COVL). Rated on a scale from 1 to 5.
• FLOPs — range: other
  • Computational complexity measured by counting floating-point operations required to process a single 2-second, 16 kHz speech sample on a GPU.

Input / output format

Input: Noisy monaural speech waveform sampled at 16 kHz, processed using an FFT length of 510 and a hop size of 100.

Output: Enhanced monaural speech waveform (16 kHz).

Scoring recipe

def score(predictions, gold):
    pesq = pesq(ref=gold, deg=predictions, fs=16000)
    stoi = stoi(ref=gold, deg=predictions, fs=16000)
    mos = mos_predict(ref=gold, deg=predictions)
    flops = count_flops(model, input_len=2*16000)
    return {'PESQ': pesq, 'STOI': stoi, 'MOS': mos, 'FLOPs': flops}

Common pitfalls

• Test set SNR distribution is fixed at specific points (-5, 0, 5, 10, 15 dB) rather than a continuous range, unlike the training set.
• FLOPs are calculated on a fixed 2-second, 16 kHz sample on GPU, not averaged over the full test set or reported in standard units (e.g., GFLOPs).
• MOS metrics (CSIG, CBAK, COVL) are algorithmic predictions of subjective scores, not actual human ratings, and require specific reference-based implementations.

Evidence (verbatim from paper)

To comprehensively assess the performance of speech enhancement models, we employ three widely adopted objective and subjective metrics. (1) Perceptual Evaluation of Speech Quality (PESQ) (Rix et al., [2001]) is designed to evaluate the perceptual quality of enhanced speech by comparing it to the clean reference signal, with a score range from -0.5 to 4.5; (2) Short-Time Objective Intelligibility (STOI) (Taal et al., [2011]) measures the proportion of intelligible content in speech signals by analyzing time-frequency correlations, with a score range from 0 to 1; (3) Mean Opinion Score (MOS) metrics provide subjective yet highly informative assessments of speech quality from a human listener’s perspective. These include signal distortion prediction (CSIG), background noise interference prediction (CBAK), and overall speech quality prediction (COVL) (Hu and Loizou, [2008]), all rated from 1 to 5.

Citation

@misc{wang2025lort,
  title={LORT: Locally Refined Convolution and Taylor Transformer for Monaural Speech Enhancement},
  author={Wang et al. (2025)},
  year={2025},
  note={arXiv:2509.23832}
}

• arXiv: 2509.23832