# T2i Deanonymization Eval

> Evaluates the ability to deanonymize text-to-image models by identifying which model generated a given image, exploiting model-specific visual signatures in embedding space. Use when the user wants to benchmark on T2I Leaderboard Prompts, or asks about evaluating this task. Reports Top-1 accuracy.

- Skill: `qhjqhj00/t2i-deanonymization-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/t2i-deanonymization-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/t2i-deanonymization-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/t2i-deanonymization-eval

---


# t2i-deanonymization-eval

> Identifying Models Behind Text-to-Image Leaderboards — Naseh et al. (2026) (arXiv:2601.09647, 2026)

## What this evaluates

Evaluates the ability to deanonymize text-to-image models by identifying which model generated a given image, exploiting model-specific visual signatures in embedding space.

## Datasets

- **T2I Leaderboard Prompts** — total 280; splits: test (280)

## Metrics

- `Top-1 accuracy` **(primary)** — range: [0, 1]
  - Fraction of test images for which the correct generating model appears within the top-k nearest centroids in the image embedding space.

## Input / output format

**Input**: Generated images from text-to-image models, optionally paired with the text prompt used to generate them.

**Output**: Predicted model identifier (for k-NN classification) or binary decision (for one-vs-rest thresholding).

## Scoring recipe

```python
def topk_accuracy(predictions, gold, k=1):
    correct = sum(1 for p, g in zip(predictions, gold) if g in p[:k])
    return correct / len(gold)
```

## Common pitfalls

- The main k-NN evaluation assumes the adversary can query all model APIs to build centroids, though a one-vs-rest variant relaxes this.
- Results are averaged over 5 randomized prompt-model assignments to account for generation variability, which must be replicated for fair comparison.
- Performance is highly sensitive to the choice of image encoder (e.g., CLIP vs. SigLIP), as noted in the ablation study.

## Evidence (verbatim from paper)

> We evaluate all attacks using top-1, top-2, and top-3 identification accuracy. These metrics quantify the fraction of test images for which the correct generating model appears within the top-k nearest centroids.

## Citation

```bibtex
@misc{naseh2026identifying,
  title={Identifying Models Behind Text-to-Image Leaderboards},
  author={Naseh et al. (2026)},
  year={2026},
  note={arXiv:2601.09647}
}
```

- arXiv: 2601.09647