# Geneval Eval

> geneval-eval

- Skill: `qhjqhj00/geneval-eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add qhjqhj00/geneval-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/geneval-eval/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/qhjqhj00/geneval-eval

---


# geneval-eval

> Nucleus-Image: Sparse MoE for Image Generation — Akiti et al. (2026) (arXiv:2604.12163, 2026)

## What this evaluates

Evaluates text-to-image alignment by testing whether models correctly render specified objects, attributes, and spatial relations across six categories: single object, two objects, counting, colors, position, and attribute binding.

## Datasets

- **GenEval** — total ?; splits: test (-1)

## Metrics

- `Overall` **(primary)** — range: [0, 1]
  - Mean accuracy across six categories: single object, two objects, counting, colors, position, and attribute binding.

## Input / output format

**Input**: Text prompt specifying objects, attributes, and spatial relations to be rendered.

**Output**: Generated image at 1024x1024 resolution, 50 inference steps, CFG scale 8.0.

## Scoring recipe

```python
category_scores = {cat: [] for cat in ['single', 'two', 'count', 'colors', 'position', 'attr']}
for prompt in prompts:
    img = model.generate(prompt, steps=50, cfg=8.0, res=1024)
    cat = get_category(prompt)
    correct = vqa_model.verify(img, prompt)
    category_scores[cat].append(correct)
overall = mean(mean(v) for v in category_scores.values())
return overall
```

## Common pitfalls

- Models frequently struggle with spatial position and counting categories compared to single-object or color rendering.
- Automated evaluation relies on external VQA models that may introduce measurement noise or bias.

## Evidence (verbatim from paper)

> Nucleus-Image achieves an overall score of 0.865 (reported as 0.87), matching Qwen-Image and surpassing all other reported models including GPT Image 1 High (0.84) and Seedream 3.0 (0.84).

## Citation

```bibtex
@misc{akiti2026nucleusimage,
  title={Nucleus-Image: Sparse MoE for Image Generation},
  author={Akiti et al. (2026)},
  year={2026},
  note={arXiv:2604.12163}
}
```

- arXiv: 2604.12163