At Add Eval

at-add-eval

AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plan — Xie et al. (2026) (arXiv:2604.08184, 2026)

What this evaluates

This evaluation protocol probes the robustness and generalization of audio deepfake detectors under real-world distortions and across heterogeneous audio types. It specifically tests whether models can maintain reliable binary classification performance when facing unseen generation methods, recording condition shifts, and unknown audio categories without relying on type-specific labels.

Datasets

• AT-ADD Challenge Dataset — total ?; splits: train (-1), development (-1), test (-1)

Metrics

• real/fake prediction (primary) — range: [0, 1]
  • Standard binary classification accuracy: the proportion of correctly classified real vs. fake audio clips out of the total evaluation set. Calculated as (TP + TN) / (TP + TN + FP + FN).

Input / output format

Input: Raw audio clip (speech utterance for Track 1; heterogeneous audio type for Track 2).

Output: Binary label indicating 'real' or 'fake'.

Scoring recipe

def calculate_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)

Common pitfalls

• Using external training data is strictly forbidden under the closed challenge setting.
• Assuming audio-type labels (speech, singing, music, etc.) are available at test time in Track 2.
• Treating signal distortions (compression, resampling, pitch shifting) or replay attacks as changing the ground-truth real/fake label.

Evidence (verbatim from paper)

Given an input speech utterance, participants are required to predict whether the input is real or fake. In this task, fake refers specifically to deepfake speech generated using deep neural network-based methods, while real refers to non-deepfake speech. It should be noted that signal distortions or transformations, such as compression, resampling, speed perturbation, and pitch shifting, as well as replay-based attacks, do not change the original real/fake label in this task.

Citation

@misc{xie2026atadd,
  title={AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plan},
  author={Xie et al. (2026)},
  year={2026},
  note={arXiv:2604.08184}
}

• arXiv: 2604.08184