qhjqhj00 geneval-t2i
Generative Refinement Networks for Visual Synthesis — Jian Han et al. (arXiv:2604.13030, 2026)
What this evaluates
Evaluates text-to-image generation by measuring how accurately models follow complex prompts with multiple objects, attributes, and spatial constraints.
Datasets
- GenEval — total ?; splits: test (-1)
Metrics
GenEval Overall(primary) — range: [0, 1]- Average accuracy across sub-tasks: Single Object, Two Object, Count, Colors, Position, and Color Attributes. Ranges from 0 to 1.
Input / output format
Input: Text prompt specifying object count, colors, and spatial relationships
Output: Generated image
Scoring recipe
scores = []
for prompt in geneval_prompts:
img = model.generate(prompt)
match = detector.check_constraints(img, prompt)
scores.append(match)
overall = mean(scores)
Common pitfalls
- Prompt rewriting is applied before generation, which can artificially boost scores.
- Evaluation relies on an external object/attribute detector that may have its own failure modes.
Evidence (verbatim from paper)
our model, augmented with a re-writer, achieves an overall score of 0.76 on the GenEval benchmark [ghosh2024geneval].
Citation
@misc{han2026generativerefinement,
title={Generative Refinement Networks for Visual Synthesis},
author={Jian Han et al.},
year={2026},
note={arXiv:2604.13030}
}
- arXiv: 2604.13030