# D Rep Eval

> Evaluates a model's ability to detect and quantify the degree of replication between an original image and a diffusion-generated replica. It probes continuous replication level prediction rather than binary copy detection, measuring how well predicted scores align with manually annotated replication levels. Use when the user wants to benchmark on D-Rep, or asks about evaluating this task. Reports PCC.

- Skill: `qhjqhj00/d-rep-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/d-rep-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/d-rep-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/d-rep-eval

---


# d-rep-eval

> Image Copy Detection for Diffusion Models — Wang et al. (2024) (arXiv:2409.19952, 2024)

## What this evaluates

Evaluates a model's ability to detect and quantify the degree of replication between an original image and a diffusion-generated replica. It probes continuous replication level prediction rather than binary copy detection, measuring how well predicted scores align with manually annotated replication levels.

## Datasets

- **D-Rep** — total 40000; splits: test (-1)

## Metrics

- `PCC` **(primary)** — range: percent
  - Pearson Correlation Coefficient between predicted replication scores and ground truth levels. Computed on cosine similarities scaled by a granularity factor N.
- `RD` — range: percent
  - Replication Deviation, defined as the mean absolute deviation between predicted and ground truth scores. Also computed on cosine similarities scaled by a granularity factor N.

## Input / output format

**Input**: Image pairs (original, replica) or extracted image features. Models compute cosine similarity between pair features.

**Output**: Predicted replication score/level (continuous or discrete) for each image pair.

## Scoring recipe

```python
def compute_metrics(predictions, ground_truth, N):
    scaled_preds = predictions * N
    scaled_gt = ground_truth * N
    pcc = pearsonr(scaled_preds, scaled_gt)
    rd = mean(abs(scaled_preds - scaled_gt))
    return pcc, rd
```

## Common pitfalls

- PCC and RD are computed on cosine similarities scaled by a granularity factor N, not raw feature distances.
- RD measures continuous deviation from ground truth levels rather than binary classification accuracy.
- Generalization tests on external diffusion models use only 100 manually labeled pairs per model, not the full D-Rep benchmark.

## Evidence (verbatim from paper)

> We employ these models as feature extractors and calculate the cosine similarity between pairs of image features (except for GPT-4V Turbo [44], see Section D in the Appendix for the implementation of it). For the computation of PCC and RD, we adjust the granularity by scaling the computed cosine similarities by a factor of $N$.

## Citation

```bibtex
@misc{wang2024imagecopydetection,
  title={Image Copy Detection for Diffusion Models},
  author={Wang et al. (2024)},
  year={2024},
  note={arXiv:2409.19952}
}
```

- arXiv: 2409.19952