# Wan Image Understanding Eval

> Evaluates the model's multimodal and text understanding capabilities across a suite of standard academic benchmarks covering visual question answering, reasoning, hallucination detection, and text-based reasoning. Use when the user wants to benchmark on MMMU, MMStar, MathVista, HalluBench, MMBench, OCRBench, AI2D, AIME, GPQA, HLE, LCBV6, or asks about evaluating this task. Reports average score.

- Skill: `qhjqhj00/wan-image-understanding-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/wan-image-understanding-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/wan-image-understanding-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/wan-image-understanding-eval

---


# wan-image-understanding-eval

> Wan-Image: Pushing the Boundaries of Generative Visual Intelligence — Mao et al. (2026) (arXiv:2604.19858, 2026)

## What this evaluates

Evaluates the model's multimodal and text understanding capabilities across a suite of standard academic benchmarks covering visual question answering, reasoning, hallucination detection, and text-based reasoning.

## Datasets

- **MMMU** — total ?; splits: test (-1)
- **MMStar** — total ?; splits: test (-1)
- **MathVista** — total ?; splits: test (-1)
- **HalluBench** — total ?; splits: test (-1)
- **MMBench** — total ?; splits: test (-1)
- **OCRBench** — total ?; splits: test (-1)
- **AI2D** — total ?; splits: test (-1)
- **AIME** — total ?; splits: test (-1)
- **GPQA** — total ?; splits: test (-1)
- **HLE** — total ?; splits: test (-1)
- **LCBV6** — total ?; splits: test (-1)

## Metrics

- `average score` **(primary)** — range: [0, 100]
  - Percentage of correctly answered questions or normalized benchmark score as reported by the official benchmark leaderboards. Averages are computed across selected benchmarks.

## Input / output format

**Input**: Image and/or text prompts/questions provided by the benchmark.

**Output**: Textual answers or multiple-choice selections.

## Scoring recipe

```python
def compute_average_score(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if normalize(p) == normalize(g))
    return (correct / len(gold)) * 100

scores = [compute_average_score(preds, golds) for preds, golds in dataset_splits]
avg_score = sum(scores) / len(scores)
```

## Common pitfalls

- Scores are reported as averages across benchmarks rather than per-dataset in the main text.
- The table uses '–' for missing indicators, which should be excluded from averages.
- Comparison includes both understanding-only and unified models, so direct comparison requires matching model scale/type.

## Evidence (verbatim from paper)

> For Image Understanding, we selected 7 classical multi-modal benchmarks to assess the model’s performance, including MMMU, MMStar, MathVista, HalluBench, MMBench, OCRBench, and AI2D. The results shown in Table 4 demonstrate that our model exhibits superior instruct and thinking capabilities: in the instruct mode, it achieves an average score 1.6 points higher than the baseline and remains comparable to other open-source understanding-only instruct models. Concurrently, in thinking mode, our model outperforms the baseline by 5.4 points and exceeds the performance of other open-source understanding-only thinking models. Compared to other unified models, our approach maintains a distinct and absolute lead. Regarding Text Understanding, our model demonstrates significant improvements over the showcased understanding-only models in both instruct and thinking modes on four common benchmarks: AIME, GPQA, HLE, and LCBV6.

## Citation

```bibtex
@misc{mao2026wanimage,
  title={Wan-Image: Pushing the Boundaries of Generative Visual Intelligence},
  author={Mao et al. (2026)},
  year={2026},
  note={arXiv:2604.19858}
}
```

- arXiv: 2604.19858