# Omnimodal QA Eval

> This benchmark probes an agent's ability to perform long-horizon question answering over audio-video streams. It specifically tests fine-grained multimodal retrieval, multi-turn tool calling, and budget-aware reasoning when key evidence is scattered across time. Use when the user wants to benchmark on OmniVideoBench, WorldSense, Daily-Omni, or asks about evaluating this task. Reports accuracy.

- Skill: `qhjqhj00/omnimodal-qa-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/omnimodal-qa-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/omnimodal-qa-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/omnimodal-qa-eval

---


# omnimodal-qa-eval

> OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answering — Zhu et al. (2026) (arXiv:2602.03707, 2026)

## What this evaluates

This benchmark probes an agent's ability to perform long-horizon question answering over audio-video streams. It specifically tests fine-grained multimodal retrieval, multi-turn tool calling, and budget-aware reasoning when key evidence is scattered across time.

## Datasets

- **OmniVideoBench** — total ?; splits: test (-1)
- **WorldSense** — total ?; splits: test (-1)
- **Daily-Omni** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly answered questions. The paper reports accuracy on each ability subset and averages them to measure overall performance across heterogeneous reasoning skills.

## Input / output format

**Input**: Long audio-video streams paired with natural language questions.

**Output**: Textual answers to the questions.

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p.strip().lower() == g.strip().lower())
    return (correct / len(gold)) * 100
```

## Common pitfalls

- Averaging accuracy across heterogeneous ability subsets may mask performance on specific reasoning skills.
- The exact answer verification method (string match vs. LLM judge) follows each benchmark's official protocol but is not explicitly detailed in the main text.
- Performance varies significantly with the retrieval budget (number of clips) and RL training state, so results are not static.

## Evidence (verbatim from paper)

> For these datasets, we report accuracy on each ability subset and their average score, which measures overall performance across heterogeneous reasoning skills.

## Citation

```bibtex
@misc{zhu2026omniragagent,
  title={OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answering},
  author={Zhu et al. (2026)},
  year={2026},
  note={arXiv:2602.03707}
}
```

- arXiv: 2602.03707