Fastturn Turn Detection Eval

fastturn-turn-detection-eval

FastTurn: Unifying Acoustic and Streaming Semantic Cues for Low-Latency and Robust Turn Detection — Wang et al. (2026) (arXiv:2604.01897, 2026)

What this evaluates

Evaluates a full-duplex turn detection model's ability to classify speech turn states (complete, incomplete, backchannel, wait) in real-time streaming conditions, measuring robustness to acoustic ambiguity, noise, and speech overlap.

Datasets

• FastTurn test set — total ?; splits: test (-1)
• Easy Turn — total 800; splits: test (800)
• Smart Turn — total ?; splits: test (-1)

Metrics

• Accuracy (primary) — range: percent
  • Accuracy = (TP + TN) / (TP + TN + FP + FN). Measures the proportion of correctly predicted turn states out of all predictions.
• Miss Rate — range: percent
  • Miss Rate = FN / (TP + FN). Measures the proportion of actual positive turn states that were incorrectly predicted as negative.
• False Alarm Rate — range: percent
  • False Alarm Rate = FP / (FP + TN). Measures the proportion of actual negative turn states that were incorrectly predicted as positive.

Input / output format

Input: Streaming acoustic features extracted via a Conformer encoder, combined with early CTC prompts processed through an LLM adapter. The model receives fused acoustic and semantic representations per time step.

Output: A categorical turn-state label (Complete, Incomplete, Backchannel, or Wait) or a binary turn-boundary decision per inference step.

Scoring recipe

tp = sum(1 for p, g in zip(preds, gold) if p == g == 1)
tn = sum(1 for p, g in zip(preds, gold) if p == g == 0)
fp = sum(1 for p, g in zip(preds, gold) if p == 1 and g == 0)
fn = sum(1 for p, g in zip(preds, gold) if p == 0 and g == 1)
accuracy = (tp + tn) / (tp + tn + fp + fn)
miss_rate = fn / (tp + fn)
false_alarm_rate = fp / (fp + tn)
return accuracy, miss_rate, false_alarm_rate

Common pitfalls

• The large ASR corpora (AISHELL, LibriSpeech, GigaSpeech, etc.) are only used for pre-training feature learning, not for turn detection evaluation. Evaluation is strictly on FastTurn, Easy Turn, and Smart Turn sets.
• Miss Rate and False Alarm Rate are defined specifically for turn-state classification (FN/TP+FN and FP/FP+TN), not as standard precision/recall or F1 scores, which can cause confusion when comparing to other dialogue benchmarks.
• Latency is reported in milliseconds but varies significantly across models due to architectural differences (e.g., Smart Turn's simplified design vs. FastTurn's unified fusion), making direct latency comparisons without accuracy context misleading.

Evidence (verbatim from paper)

To evaluate model performance in full-duplex conversational scenarios, we employ three primary metrics: Accuracy, Miss Rate, and False Alarm Rate. These are derived from turn-state classification results, where True Positives (TP) and True Negatives (TN) denote correct predictions, and False Positives (FP) and False Negatives (FN) denote errors. | Accuracy = (TP + TN) / (TP + TN + FP + FN). | Miss Rate = FN / (TP + FN). | False Alarm Rate = FP / (FP + TN).

Citation

@misc{wang2026fastturn,
  title={FastTurn: Unifying Acoustic and Streaming Semantic Cues for Low-Latency and Robust Turn Detection},
  author={Wang et al. (2026)},
  year={2026},
  note={arXiv:2604.01897}
}

• arXiv: 2604.01897