# Veu Bench Eval

> veu-bench-eval

- Skill: `qhjqhj00/veu-bench-eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add qhjqhj00/veu-bench-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/veu-bench-eval/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/qhjqhj00/veu-bench-eval

---


# veu-bench-eval

> VEU-Bench: Towards Comprehensive Understanding of Video Editing — Li et al. (2025) (arXiv:2504.17828, 2025)

## What this evaluates

VEU-Bench evaluates a model's ability to understand video editing by probing 19 fine-grained tasks across 10 dimensions (e.g., shot size, cut types, transitions) and three cognitive stages: recognition, reasoning, and judging. It tests whether models can identify editing components, infer their functions, and judge their effects in video content.

## Datasets

- **VEU-Bench** — total 50000; splits: train (-1), val (-1), test (-1)

## Metrics

- `Scoremc` — range: [1, 5]
  - Average score on the recognition task dimension, rated on a 1-5 scale based on human or LLM evaluation of answer quality.
- `Scoreoe` — range: [1, 5]
  - Average score on the reasoning and judging task dimensions, rated on a 1-5 scale based on human or LLM evaluation of answer quality.
- `Scoreall` **(primary)** — range: [1, 5]
  - Overall average score across all 19 tasks and 10 dimensions, rated on a 1-5 scale.

## Input / output format

**Input**: Video frames sampled at 1 fps (maximum 64 frames) paired with a text prompt containing a question about a specific video editing element (e.g., shot size, cut type, transition function).

**Output**: Natural language text response answering the prompt, identifying or reasoning about the specified editing component.

## Scoring recipe

```python
scores = []
for answer in predictions:
    score = rater.evaluate(answer, gold_reference)  # Returns 1-5
    scores.append(score)
metric_value = sum(scores) / len(scores)
return metric_value
```

## Common pitfalls

- Uniform frame sampling (1 fps) limits models' ability to perceive motion speed and dynamic changes, sometimes causing performance worse than random guessing.
- Models frequently confuse intra-frame content with inter-frame editing effects, leading to artificially high recognition scores but poor reasoning/judging performance.
- LLM evaluators tend to bias scores toward visual factual content rather than correctly judging editing intent unless pattern-matching regularization is applied.

## Evidence (verbatim from paper)

> Meanwhile, Oscars gains 28.3% higher performance in Scoreall compared to SOTA open-source model LLaVA-OneVision*[[19]]*. In the more challenging dimensions cut and transition, Oscars demonstrates significantly better performance than the open-source Vid-LLM state-of-the-art, with improvements of 12.2% and 10.9% on Scoremc and Scoreoe respectively, showing exceeding video editing understanding ability.

## Citation

```bibtex
@misc{li2025veubench,
  title={VEU-Bench: Towards Comprehensive Understanding of Video Editing},
  author={Li et al. (2025)},
  year={2025},
  note={arXiv:2504.17828}
}
```

- arXiv: 2504.17828