L2 Arctic Accent Eval

l2-arctic-accent-eval

PHONOS: PHOnetic Neutralization for Online Streaming Applications — Quamer et al. (2026) (arXiv:2603.27001, 2026)

What this evaluates

Evaluates a streaming foreign accent conversion system's ability to neutralize non-native pronunciation while preserving speaker identity. The protocol uses self-reconstruction mode and compares synthesized outputs against offline-generated golden speaker utterances as a reference baseline.

Datasets

• L2-ARCTIC (Indian subset) — total 400; splits: test (400)

Metrics

• non-native accent confidence (primary) — range: percent
  • Measures the classifier's confidence that the synthesized speech retains non-native accent characteristics. Evaluated as the percentage reduction in confidence scores relative to the original source utterances.

Input / output format

Input: 16 kHz audio utterance with ground truth transcription, conditioned on a speaker embedding extracted from the same source utterance.

Output: Synthesized neutralized speech audio.

Scoring recipe

def score(predictions, gold):
    # predictions: list of synthesized audio files
    # gold: list of corresponding source utterances
    conf_scores = []
    for pred, src in zip(predictions, gold):
        # Run accent classifier on both
        pred_conf = classifier.predict_confidence(pred)
        src_conf = classifier.predict_confidence(src)
        conf_scores.append((src_conf - pred_conf) / src_conf)
    return mean(conf_scores) * 100  # percentage reduction

Common pitfalls

• No open-source baselines available; evaluation relies solely on golden-speaker reference upper bound
• Self-reconstruction mode conditions the model on the exact same speaker embedding as the source utterance
• Golden-speaker utterances are generated offline and discarded after training, serving only as a static reference during evaluation

Evidence (verbatim from paper)

achieving an 81% reduction in non-native accent confidence via both classifier and human ratings, with end-to-end GPU latency under 241ms and reduced speaker linkability in embedding space.

Citation

@misc{quamer2026phonos,
  title={PHONOS: PHOnetic Neutralization for Online Streaming Applications},
  author={Quamer et al. (2026)},
  year={2026},
  note={arXiv:2603.27001}
}

• arXiv: 2603.27001