Kvbench Eval

kvbench-eval

Knowledge Visualization: A Benchmark and Method for Knowledge-Intensive Text-to-Image Generation — Ran Zhao et al. (2026) (arXiv:2604.22302, 2026)

What this evaluates

Evaluates text-to-image models on knowledge-intensive generation across six high-school academic subjects and two languages. It probes scientific fidelity, logical reasoning, symbolic precision, and multilingual robustness using textbook-derived prompts and atomic checklist verification.

Datasets

• KVBench — total ?; splits: test (-1); repo https://github.com/zhaoran66/KVBench

Metrics

• performance_score (primary) — range: percent
  • Percentage of generated images that pass an atomic checklist of domain-specific knowledge constraints and visual elements. Calculated as (number of passing instances / total instances) * 100.

Input / output format

Input: Text prompts (Brief Caption or Detailed Caption) describing academic concepts, provided in either Chinese or English.

Output: A single generated image per prompt.

Scoring recipe

def compute_performance_score(predictions, gold_checklists):
    passed = 0
    for pred_img, checklist in zip(predictions, gold_checklists):
        if all(check_constraint(pred_img, c) for c in checklist):
            passed += 1
    return (passed / len(predictions)) * 100

Common pitfalls

• Detailed captions can degrade performance for weaker open-source models due to poor instruction-following and interference.
• Models show significant multilingual fragility, with English prompts consistently outperforming Chinese ones.
• Performance gaps are largest in reasoning-heavy subjects (Math, Physics) requiring symbolic precision and multi-step logic.

Evidence (verbatim from paper)

Table 2 presents a comprehensive comparison of model performance across six academic disciplines and two languages (“zh” for Chinese and “en” for English.

Citation

@misc{zhao2026knowledgevisualization,
  title={Knowledge Visualization: A Benchmark and Method for Knowledge-Intensive Text-to-Image Generation},
  author={Ran Zhao et al. (2026)},
  year={2026},
  note={arXiv:2604.22302}
}

• arXiv: 2604.22302