# Enem Vision Eval

> Evaluates multimodal and text-only language models on Brazilian university admission exams (ENEM), specifically probing their ability to comprehend visual information, interpret tables/figures, and perform mathematical reasoning in a multiple-choice format. Use when the user wants to benchmark on ENEM 2022/2023, or asks about evaluating this task. Reports accuracy.

- Skill: `qhjqhj00/enem-vision-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/enem-vision-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/enem-vision-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/enem-vision-eval

---


# enem-vision-eval

> Evaluating GPT-4's Vision Capabilities on Brazilian University Admission Exams — Pires et al. (2023) (arXiv:2311.14169, 2023)

## What this evaluates

Evaluates multimodal and text-only language models on Brazilian university admission exams (ENEM), specifically probing their ability to comprehend visual information, interpret tables/figures, and perform mathematical reasoning in a multiple-choice format.

## Datasets

- **ENEM 2022/2023** — total ?; splits: test (-1); repo https://github.com/piresramon/gpt-4-enem

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly answered multiple-choice questions. Calculated as (number of correct predictions / total number of questions) * 100.

## Input / output format

**Input**: Multiple-choice question text with five options (A–E), optionally accompanied by an image or a human-generated textual caption. Prompts are structured as 3-shot Chain-of-Thought examples.

**Output**: Step-by-step reasoning (Chain-of-Thought) followed by the final selected answer choice.

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p.strip().upper() == g.strip().upper())
    return (correct / len(gold)) * 100
```

## Common pitfalls

- Failing to replicate the exact 3-shot Chain-of-Thought prompt template, which is critical for model performance.
- Confusing the three experimental conditions (no image, direct image, caption) that isolate different vision-language capabilities.
- Assuming human-generated captions fully preserve the visual information required for complex mathematical or table-based questions.

## Evidence (verbatim from paper)

> We performed three experiments to analyze multiple-choice questions: without images, with images, and with captions. Each experiment is described hereafter. 1) Without Images: In this experiment, we analyzed multiple-choice questions excluding any visual elements. ... 2) With Images: For this experiment, we aim to evaluate multimodal LMs by embedding the images directly into the question body. ... 3) With Captions: In this experiment, our focus was on assessing textual LMs by replacing images with textual descriptions containing all essential information necessary for understanding the question.

## Citation

```bibtex
@misc{pires2023enem,
  title={Evaluating GPT-4's Vision Capabilities on Brazilian University Admission Exams},
  author={Pires et al. (2023)},
  year={2023},
  note={arXiv:2311.14169}
}
```

- arXiv: 2311.14169