# Image Synthesis Eval

> Evaluates the visual fidelity and text-image alignment of generated images. It measures realism and distribution matching using FID, and semantic alignment using CLIP scores. Use when the user wants to benchmark on COCO-2014, or asks about evaluating this task. Reports FID (CLIP features).

- Skill: `qhjqhj00/image-synthesis-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/image-synthesis-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/image-synthesis-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/image-synthesis-eval

---


# image-synthesis-eval

> Scaling Rectified Flow Transformers for High-Resolution Image Synthesis — Esser et al. (2024) (arXiv:2403.03206, 2024)

## What this evaluates

Evaluates the visual fidelity and text-image alignment of generated images. It measures realism and distribution matching using FID, and semantic alignment using CLIP scores.

## Datasets

- **COCO-2014** — total ?; splits: val (-1)

## Metrics

- `FID (CLIP features)` **(primary)** — range: [0, ∞) (lower is better)
  - Fréchet Inception Distance computed on CLIP image features to measure the distance between real and generated image distributions. Lower values indicate better quality.
- `CLIP score` — range: [0, 1] (higher is better)
  - Cosine similarity between CLIP image and text embeddings. Higher values indicate better text-image alignment.

## Input / output format

**Input**: Text prompt (for generation) and real images (for FID calculation).

**Output**: Generated RGB image.

## Scoring recipe

```python
def compute_metrics(gen_images, real_images, prompts):
    clip_gen = clip_model.encode(gen_images)
    clip_real = clip_model.encode(real_images)
    fid = frechet_distance(clip_gen, clip_real)
    clip_scores = cosine_similarity(clip_gen, clip_model.encode(prompts))
    return fid, clip_scores.mean()
```

## Common pitfalls

- FID is calculated on CLIP features, not the standard Inception v3 features.
- Lower FID is better, while higher CLIP score is better.
- Evaluation is strictly performed on the COCO-2014 validation split, not the training data.

## Evidence (verbatim from paper)

> We calculate the FID on CLIP features as proposed by (Sauer et al., 2021). All metrics are evaluated on the COCO-2014 validation split (Lin et al., 2014).

## Citation

```bibtex
@misc{esser2024scalingrectifiedflow,
  title={Scaling Rectified Flow Transformers for High-Resolution Image Synthesis},
  author={Esser et al. (2024)},
  year={2024},
  note={arXiv:2403.03206}
}
```

- arXiv: 2403.03206