qhjqhj00 dpg-bench-eval
Nucleus-Image: Sparse MoE for Image Generation — Akiti et al. (2026) (arXiv:2604.12163, 2026)
What this evaluates
Evaluates dense prompt following on multi-requirement prompts by decomposing them into dependency-structured VQA checks spanning entity presence, attributes, relations, and counts.
Datasets
- DPG-Bench — total 1065; splits: test (1065)
Metrics
Overall(primary) — range: [0, 100]- Mean score across five subcategories: Global, Entity, Attribute, Relation, and Other.
Input / output format
Input: Text prompt containing multiple dense requirements and constraints.
Output: Generated image at 1024x1024 resolution, 50 inference steps, CFG scale 8.0.
Scoring recipe
scores = []
for prompt in prompts:
img = model.generate(prompt, steps=50, cfg=8.0, res=1024)
checks = decompose_prompt_into_vqa(prompt)
correct = sum(vqa_model.verify(img, check) for check in checks)
scores.append(correct / len(checks))
return mean(scores) * 100
Common pitfalls
- Global consistency scores are often lower than fine-grained entity/attribute scores, indicating holistic scene coherence is harder to achieve.
- Evaluation requires careful decomposition of prompts into structured VQA checks to avoid missing implicit constraints.
Evidence (verbatim from paper)
Nucleus-Image achieves the highest overall score of 88.79, narrowly outperforming Qwen-Image (88.32) and Seedream 3.0 (88.27).
Citation
@misc{akiti2026nucleusimage,
title={Nucleus-Image: Sparse MoE for Image Generation},
author={Akiti et al. (2026)},
year={2026},
note={arXiv:2604.12163}
}
- arXiv: 2604.12163