qhjqhj00 t2i-deanonymization-eval
Identifying Models Behind Text-to-Image Leaderboards — Naseh et al. (2026) (arXiv:2601.09647, 2026)
What this evaluates
Evaluates the ability to deanonymize text-to-image models by identifying which model generated a given image, exploiting model-specific visual signatures in embedding space.
Datasets
- T2I Leaderboard Prompts — total 280; splits: test (280)
Metrics
Top-1 accuracy(primary) — range: [0, 1]- Fraction of test images for which the correct generating model appears within the top-k nearest centroids in the image embedding space.
Input / output format
Input: Generated images from text-to-image models, optionally paired with the text prompt used to generate them.
Output: Predicted model identifier (for k-NN classification) or binary decision (for one-vs-rest thresholding).
Scoring recipe
def topk_accuracy(predictions, gold, k=1):
correct = sum(1 for p, g in zip(predictions, gold) if g in p[:k])
return correct / len(gold)
Common pitfalls
- The main k-NN evaluation assumes the adversary can query all model APIs to build centroids, though a one-vs-rest variant relaxes this.
- Results are averaged over 5 randomized prompt-model assignments to account for generation variability, which must be replicated for fair comparison.
- Performance is highly sensitive to the choice of image encoder (e.g., CLIP vs. SigLIP), as noted in the ablation study.
Evidence (verbatim from paper)
We evaluate all attacks using top-1, top-2, and top-3 identification accuracy. These metrics quantify the fraction of test images for which the correct generating model appears within the top-k nearest centroids.
Citation
@misc{naseh2026identifying,
title={Identifying Models Behind Text-to-Image Leaderboards},
author={Naseh et al. (2026)},
year={2026},
note={arXiv:2601.09647}
}
- arXiv: 2601.09647