# Mia Eval

> Evaluates the reasoning, tool-use, and memory-augmented planning capabilities of agents on complex multi-hop QA and visual question answering tasks. It probes how well models leverage episodic memory, test-time learning, and reflection to improve answer accuracy over iterative search trajectories. Use when the user wants to benchmark on FVQA-test, InfoSeek, MMSearch, SimpleVQA, LiveVQA, In-house 1, In-house 2, HotpotQA, 2WikiMultiHopQA, SimpleQA, GAIA (text-only subset), or asks about evaluating this task. Reports accuracy.

- Skill: `qhjqhj00/mia-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/mia-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/mia-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/mia-eval

---


# mia-eval

> Memory Intelligence Agent — Jingyang Qiao et al. (2026) (arXiv:2604.04503, 2026)

## What this evaluates

Evaluates the reasoning, tool-use, and memory-augmented planning capabilities of agents on complex multi-hop QA and visual question answering tasks. It probes how well models leverage episodic memory, test-time learning, and reflection to improve answer accuracy over iterative search trajectories.

## Datasets

- **FVQA-test** — total ?; splits: test (-1)
- **InfoSeek** — total ?; splits: test (-1)
- **MMSearch** — total ?; splits: test (-1)
- **SimpleVQA** — total ?; splits: test (-1)
- **LiveVQA** — total ?; splits: test (-1)
- **In-house 1** — total ?; splits: test (-1)
- **In-house 2** — total ?; splits: test (-1)
- **HotpotQA** — total ?; splits: test (-1)
- **2WikiMultiHopQA** — total ?; splits: test (-1)
- **SimpleQA** — total ?; splits: test (-1)
- **GAIA (text-only subset)** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of questions answered correctly, determined by an external LLM judge (Qwen3-32B) comparing the model's final output to the ground truth label.

## Input / output format

**Input**: Natural language questions, optionally accompanied by images, along with access to external search tools (wiki25 for text-to-text search, Serper for image-to-image and text-to-text search).

**Output**: A short, precise final answer string generated after multi-turn tool calling and reasoning.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_answers):
    correct = 0
    for pred, gold in zip(predictions, gold_answers):
        # Qwen3-32B LLM Judger evaluates correctness per Appendix 13
        is_correct = llm_judge_check(pred, gold['answer'])
        if is_correct:
            correct += 1
    return (correct / len(predictions)) * 100
```

## Common pitfalls

- Long-context memory methods (e.g., RAG, Mem0) often underperform simpler baselines due to noise introduced by extended contexts.
- Correctness is determined entirely by an external LLM judge (Qwen3-32B), so results depend heavily on the judge's prompt template and model version.
- Tool usage is unconstrained, leading to high variance in reasoning paths and call counts across different models and runs.

## Evidence (verbatim from paper)

> MIA achieves the highest overall performance among open-source models, reaching an average accuracy of 53.6. Compared to the previous best memory-based method, MIA improves the average accuracy by 5.5, including specific increases of 3.3 on FVQA-test, 6.4 on the multi-hop task LiveVQA, and an impressive 9.1 on the highly challenging custom task In-house 1.

## Citation

```bibtex
@misc{qiao2026memoryintelligenceagent,
  title={Memory Intelligence Agent},
  author={Jingyang Qiao et al. (2026)},
  year={2026},
  note={arXiv:2604.04503}
}
```

- arXiv: 2604.04503