# Anetqa Eval

> Evaluates fine-grained compositional reasoning over untrimmed videos by requiring models to interpret spatio-temporal scene graphs and answer complex questions involving attributes, actions, and temporal relationships. Use when the user wants to benchmark on ANetQA, or asks about evaluating this task. Reports accuracy.

- Skill: `qhjqhj00/anetqa-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/anetqa-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/anetqa-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/anetqa-eval

---


# anetqa-eval

> ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos — Zhou Yu et al. (2023) (arXiv:2305.02519, 2023)

## What this evaluates

Evaluates fine-grained compositional reasoning over untrimmed videos by requiring models to interpret spatio-temporal scene graphs and answer complex questions involving attributes, actions, and temporal relationships.

## Datasets

- **ANetQA** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Calculated as the percentage of correctly predicted answers out of the total number of questions. Per-type accuracies are also reported under four taxonomies: question structures, question semantics, reasoning skills, and answer types.

## Input / output format

**Input**: Untrimmed video and a natural language question.

**Output**: A single predicted answer (binary choice, open-ended text, or one of four choices: [A], [B], both, or none).

## Scoring recipe

```python
correct = 0
total = 0
for pred, gold in zip(predictions, gold_answers):
    if pred == gold:
        correct += 1
    total += 1
accuracy = (correct / total) * 100
```

## Common pitfalls

- Models may over-rely on language priors rather than visual reasoning, though balancing strategies are used to mitigate this.
- Frame sampling strategy heavily impacts performance; insufficient or non-uniform sampling often misses critical temporal cues needed for fine-grained reasoning.
- Open-ended and attribute-oriented questions are significantly harder than binary/choice questions, leading to sharp accuracy drops.

## Evidence (verbatim from paper)

> Besides the overall accuracy, we follow [[9]] to report the per-type accuracies under different taxonomies, i.e., question structures, question semantics, reasoning skills, and answer types.

## Citation

```bibtex
@misc{yu2023anetqa,
  title={ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos},
  author={Zhou Yu et al. (2023)},
  year={2023},
  note={arXiv:2305.02519}
}
```

- arXiv: 2305.02519