points-long-video-image-eval
POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs — Haicheng Wang et al. (arXiv:2604.11627, 2026)
What this evaluates
Evaluates multimodal large language models on fine-grained image understanding and long-form video comprehension tasks. It measures the trade-off between visual token compression efficiency and task accuracy across diverse benchmarks.
Datasets
- MVBench — total ?; splits: test (-1)
- Video-MME — total ?; splits: test (-1)
- MLVU — total ?; splits: test (-1)
- LongVideoBench — total ?; splits: test (-1)
- MMBench — total ?; splits: test (-1)
- MMMU_val — total ?; splits: val (-1)
Metrics
accuracy (primary) — range: percent
- Percentage of correctly answered questions across each benchmark. Calculated as (number of correct predictions / total predictions) * 100. Evaluated using benchmark-specific protocols via VLMEvalKit and lmms-eval.
Input / output format
Input: Video sequences (typically 64 frames) or images paired with text prompts/questions. Visual tokens are dynamically compressed to 8, 16, or 32 per frame in Standby mode, or retained densely in Focus mode.
Output: Text responses answering the provided VQA prompts.
Scoring recipe
correct = 0
total = 0
for pred, gold in zip(predictions, gold_labels):
if pred.strip().lower() == gold.strip().lower():
correct += 1
total += 1
return (correct / total) * 100
Common pitfalls
- Confusing Standby vs Focus mode token budgets (2.5%-10% vs ~100%) and their respective performance baselines.
- Assuming standard 64-frame evaluation applies to streaming benchmarks, which use variable frame counts (e.g., 248+8, 504+8).
- Overlooking that average scores are computed across heterogeneous benchmarks with different evaluation protocols (exact match, LLM-judge, etc.).
Evidence (verbatim from paper)
We evaluate on a wide range of video benchmarks, including Opencompass video leaderboard: VideoMME, Tempcompass, MVBench, MMBench-Video, MLVU, LongVideoBench, and other commonly used video benchmarks: MovieChat1K, CG-Bench, EgoSchema, TemporalBench, Activitynet-qa, LVBench and WorldSense. We use VLMEvalKit and lmms-eval for evaluation.
Citation
@misc{wang2026pointslong,
title={POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs},
author={Haicheng Wang et al.},
year={2026},
note={arXiv:2604.11627}
}
1---2name: points-long-video-image-eval3description: Evaluates multimodal large language models on fine-grained image understanding and long-form video comprehension tasks. It measures the trade-off between visual token compression efficiency and task accuracy across diverse benchmarks. Use when the user wants to benchmark on MVBench, Video-MME, MLVU, LongVideoBench, MMBench, MMMU_val, or asks about evaluating this task. Reports accuracy.4metadata:5 source: https://skillmd.com/skills/qhjqhj00/points-long-video-image-eval6 skill_kind: dataset_eval7 source_arxiv: 2604.116278 bibtex_key: wang2026pointslong9 confidence: high10---1112# points-long-video-image-eval1314> POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs — Haicheng Wang et al. (arXiv:2604.11627, 2026)1516## What this evaluates1718Evaluates multimodal large language models on fine-grained image understanding and long-form video comprehension tasks. It measures the trade-off between visual token compression efficiency and task accuracy across diverse benchmarks.1920## Datasets2122- **MVBench** — total ?; splits: test (-1)23- **Video-MME** — total ?; splits: test (-1)24- **MLVU** — total ?; splits: test (-1)25- **LongVideoBench** — total ?; splits: test (-1)26- **MMBench** — total ?; splits: test (-1)27- **MMMU_val** — total ?; splits: val (-1)2829## Metrics3031- `accuracy` **(primary)** — range: percent32 - Percentage of correctly answered questions across each benchmark. Calculated as (number of correct predictions / total predictions) * 100. Evaluated using benchmark-specific protocols via VLMEvalKit and lmms-eval.3334## Input / output format3536**Input**: Video sequences (typically 64 frames) or images paired with text prompts/questions. Visual tokens are dynamically compressed to 8, 16, or 32 per frame in Standby mode, or retained densely in Focus mode.3738**Output**: Text responses answering the provided VQA prompts.3940## Scoring recipe4142```python43correct = 044total = 045for pred, gold in zip(predictions, gold_labels):46 if pred.strip().lower() == gold.strip().lower():47 correct += 148 total += 149return (correct / total) * 10050```5152## Common pitfalls5354- Confusing Standby vs Focus mode token budgets (2.5%-10% vs ~100%) and their respective performance baselines.55- Assuming standard 64-frame evaluation applies to streaming benchmarks, which use variable frame counts (e.g., 248+8, 504+8).56- Overlooking that average scores are computed across heterogeneous benchmarks with different evaluation protocols (exact match, LLM-judge, etc.).5758## Evidence (verbatim from paper)5960> We evaluate on a wide range of video benchmarks, including Opencompass video leaderboard: VideoMME, Tempcompass, MVBench, MMBench-Video, MLVU, LongVideoBench, and other commonly used video benchmarks: MovieChat1K, CG-Bench, EgoSchema, TemporalBench, Activitynet-qa, LVBench and WorldSense. We use VLMEvalKit and lmms-eval for evaluation.6162## Citation6364```bibtex65@misc{wang2026pointslong,66 title={POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs},67 author={Haicheng Wang et al.},68 year={2026},69 note={arXiv:2604.11627}70}71```7273- arXiv: 2604.11627