Voice Morph Threshold Eval

voice-morph-threshold-eval

VoiceMorph: How AI Voice Morphing Reveals the Boundaries of Auditory Self-Recognition — Shimizu et al. (2025) (arXiv:2510.16192, 2025)

What this evaluates

This evaluation probes auditory self-recognition boundaries by measuring how much AI voice morphing a participant can tolerate before they stop recognizing their own voice. It assesses perceptual thresholds, decision latency, and the influence of acoustic embedding distances and demographic factors on voice identity perception.

Datasets

• VoiceMorph Experimental Dataset — total ?; splits: test (-1)

Metrics

• lowess_T (primary) — range: percent
  • The percentage of voice morphing at which a participant's self-recognition rating drops below 4.0 on a 7-point Likert scale. Extracted by fitting a LOWESS curve (smoothing fraction 0.4) to ratings across morphing levels and finding the crossing point.

Input / output format

Input: Audio clips of the participant's own voice, another speaker's voice, and morphed versions at varying degrees (0% to 100%). Participants listen to each clip and provide a self-recognition rating.

Output: A 7-point Likert scale rating (1-7) for self-recognition similarity, and the response time in milliseconds for each clip.

Scoring recipe

# Input: morphing_levels (0-100), ratings (7-point Likert), response_times (ms), embedding_distances
# 1. Fit LOWESS curve to ratings across morphing levels (frac=0.4)
lowess_fit = LOWESS(morphing_levels, ratings, frac=0.4)
# 2. Find crossing point where fit drops below 4.0
lowess_T = morphing_level where lowess_fit < 4.0
# 3. Compute correlations for embedding distances vs thresholds/RT
r_threshold = pearsonr(embedding_distances, lowess_T)
r_rt = pearsonr(embedding_distances, response_times)
# 4. Apply BH-FDR correction for pairwise condition contrasts

Common pitfalls

• Threshold is defined at a rating of 4.0 on a 7-point scale, which is arbitrary and may not align with standard signal detection d-prime metrics.
• Acoustic embedding distances (e.g., Euclidean distance in latent space) do not predict perceptual thresholds, only response times; relying solely on embedding similarity will misrepresent recognition boundaries.
• Response times show a non-monotonic or weak trend with morphing level, requiring careful statistical control (e.g., FDR correction) to avoid false positives.

Evidence (verbatim from paper)

Table 2. OLS Regression Results: Voice Boundary Thresholds (lowess_T) ††: ... Dependent variable: voice boundary threshold (morphing % where self-recognition ¡ 4.0). First, we applied LOWESS (Locally Weighted Scatterplot Smoothing) with smoothing fraction 0.4 to extract individual perceptual thresholds where self-recognition ratings crossed below 4.0 on our 7-point scale.

Citation

@misc{shimizu2025voicemorph,
  title={VoiceMorph: How AI Voice Morphing Reveals the Boundaries of Auditory Self-Recognition},
  author={Shimizu et al. (2025)},
  year={2025},
  note={arXiv:2510.16192}
}

• arXiv: 2510.16192