Trace Eval

trace-eval

TRACE: Training-Free Partial Audio Deepfake Detection via Embedding Trajectory Analysis of Speech Foundation Models — Awais Khan et al. (arXiv:2604.01083, 2026)

What this evaluates

This protocol evaluates training-free partial audio deepfake detection by analyzing the temporal continuity of frozen speech foundation model embeddings. It probes a model's ability to detect splice boundaries and synthetic insertions in speech without requiring labeled training data or architectural modifications.

Datasets

• PartialSpoof — total 121461; splits: train (25380), dev (24844), eval (71237)
• HalfTruth Audio Deepfake (HAD) — total 79989; splits: train (53093), dev (17824), test (9072)
• ADD 2023 Track 2 — total 120917; splits: train (53093), dev (17824), test (50000)
• LlamaPartialSpoof — total 140616; splits: R01TTS.0.a (76228), R01TTS.0.b (64388)

Metrics

• EER (primary) — range: percent
  • Equal Error Rate: the threshold that minimizes the difference between False Acceptance Rate (FAR) and False Rejection Rate (FRR), reported as their average.
• AUC — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve, measuring the probability that a randomly chosen spoof utterance scores higher than a bonafide one.
• Transfer EER — range: percent
  • EER computed using a fixed threshold calibrated on the PartialSpoof dev set, applied directly to the target test set without target-domain data.
• Free EER — range: percent
  • EER computed using the optimal threshold found by searching over the target test set itself.

Input / output format

Input: Raw audio utterances. The system extracts frame-level embeddings at 50 Hz from frozen speech foundation models (e.g., WavLM, HuBERT, Wav2Vec 2.0, Whisper).

Output: A continuous anomaly score per utterance, derived from first-order temporal dynamics of the embedding trajectory. A binary decision (bonafide vs. partial fake) is made by comparing the score against a threshold.

Scoring recipe

def compute_eer(scores, labels):
    # scores: utterance-level anomaly scores
    # labels: 0 (bonafide), 1 (spoof)
    # Find threshold t minimizing |FAR(t) - FRR(t)|
    # FAR = P(score > t | label=0), FRR = P(score <= t | label=1)
    return (FAR(t) + FRR(t)) / 2

def compute_auc(scores, labels):
    return auc(scores, labels)

Common pitfalls

• Confusing 'Transfer EER' (fixed threshold from PartialSpoof dev) with 'Free EER' (threshold optimized on target set).
• HAD test-set bona fide labels are withheld by organizers, so evaluation must strictly use the dev split.
• Fully-fake TTS utterances in LlamaPartialSpoof lack splice boundaries, making them inherently harder for this boundary-focused method and inflating EER (~45%).

Evidence (verbatim from paper)

We report Equal Error Rate (EER, %, ↓) and Area Under the ROC Curve (AUC, ↑). For cross-dataset evaluation, we additionally report Transfer EER: the EER obtained by applying the PartialSpoof dev-calibrated threshold directly to the target test set, strictly measuring domain transfer without any target-domain data. Free EER, the EER at the optimal threshold on the target set, serves as an upper bound on in-dataset performance.

Citation

@misc{khan2026trace,
  title={TRACE: Training-Free Partial Audio Deepfake Detection via Embedding Trajectory Analysis of Speech Foundation Models},
  author={Awais Khan et al.},
  year={2026},
  note={arXiv:2604.01083}
}

• arXiv: 2604.01083