# P2g Visual Reasoning Eval

> Evaluates the visual reasoning and text-understanding capabilities of multimodal large language models (MLLMs) on high-resolution, text-rich, and general semantic images. It probes whether agent-augmented grounding improves answer accuracy compared to vanilla MLLMs and proprietary models like GPT-4V. Use when the user wants to benchmark on DocVQA, ChartVQA, GQA, SEED, MM-VET, MME, P2GB, or asks about evaluating this task. Reports VQA score.

- Skill: `qhjqhj00/p2g-visual-reasoning-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/p2g-visual-reasoning-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/p2g-visual-reasoning-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/p2g-visual-reasoning-eval

---


# p2g-visual-reasoning-eval

> Plug-and-Play Grounding of Reasoning in Multimodal Large Language Models — Chen et al. (2024) (arXiv:2403.19322, 2024)

## What this evaluates

Evaluates the visual reasoning and text-understanding capabilities of multimodal large language models (MLLMs) on high-resolution, text-rich, and general semantic images. It probes whether agent-augmented grounding improves answer accuracy compared to vanilla MLLMs and proprietary models like GPT-4V.

## Datasets

- **DocVQA** — total ?; splits: test (-1)
- **ChartVQA** — total ?; splits: test (-1)
- **GQA** — total ?; splits: test (-1)
- **SEED** — total ?; splits: test (-1)
- **MM-VET** — total ?; splits: test (-1)
- **MME** — total ?; splits: test (-1)
- **P2GB** — total ?; splits: test (-1)

## Metrics

- `VQA score` **(primary)** — range: percent
  - Standard dataset-specific evaluation metric (typically exact match or fuzzy match accuracy) used for each visual question answering benchmark. Scores are reported as percentages in the tables.

## Input / output format

**Input**: High-resolution image paired with a natural language question or instruction.

**Output**: Natural language answer string, optionally accompanied by generated reasoning and bounding box coordinates for grounded objects/texts.

## Scoring recipe

```python
def compute_vqa_score(predictions, gold_answers):
    score = 0
    for pred, golds in zip(predictions, gold_answers):
        # Standard VQA evaluation: exact match or fuzzy match (e.g., 3-gram overlap)
        if normalize_answer(pred) in [normalize_answer(g) for g in golds]:
            score += 1
    return (score / len(predictions)) * 100
```

## Common pitfalls

- Benchmark scores are highly sensitive to input resolution and context window limits; high-res images may be truncated or downsampled.
- Text-rich benchmarks require external OCR agents; OCR errors or missed text directly degrade scores.
- Including relative position vectors of grounded boxes in the prompt significantly impacts performance on structured data like charts.

## Evidence (verbatim from paper)

> Table 5: Experimental results of P2G and baselines on visual reasoning benchmarks. The performance of P2G on visual reasoning benchmarks are presented in Table 6. On text-rich visual reasoning, P2G significantly outperform baselines, including the vanilla LLaVA, by more than doubled (3× on DocVQA, 2.4× on ChartVQA)... On general visual reasoning benchmarks, P2G also enjoys a consistent improvement over LLaVA and InstrtuctBLIP, demonstrating the superiority of P2G.

## Citation

```bibtex
@misc{chen2024p2g,
  title={Plug-and-Play Grounding of Reasoning in Multimodal Large Language Models},
  author={Chen et al. (2024)},
  year={2024},
  note={arXiv:2403.19322}
}
```

- arXiv: 2403.19322