Points Long Video Image Eval

points-long-video-image-eval

POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs — Haicheng Wang et al. (arXiv:2604.11627, 2026)

What this evaluates

Evaluates multimodal large language models on fine-grained image understanding and long-form video comprehension tasks. It measures the trade-off between visual token compression efficiency and task accuracy across diverse benchmarks.

Datasets

• MVBench — total ?; splits: test (-1)
• Video-MME — total ?; splits: test (-1)
• MLVU — total ?; splits: test (-1)
• LongVideoBench — total ?; splits: test (-1)
• MMBench — total ?; splits: test (-1)
• MMMU_val — total ?; splits: val (-1)

Metrics

• accuracy (primary) — range: percent
  • Percentage of correctly answered questions across each benchmark. Calculated as (number of correct predictions / total predictions) * 100. Evaluated using benchmark-specific protocols via VLMEvalKit and lmms-eval.

Input / output format

Input: Video sequences (typically 64 frames) or images paired with text prompts/questions. Visual tokens are dynamically compressed to 8, 16, or 32 per frame in Standby mode, or retained densely in Focus mode.

Output: Text responses answering the provided VQA prompts.

Scoring recipe

correct = 0
total = 0
for pred, gold in zip(predictions, gold_labels):
    if pred.strip().lower() == gold.strip().lower():
        correct += 1
    total += 1
return (correct / total) * 100

Common pitfalls

• Confusing Standby vs Focus mode token budgets (2.5%-10% vs ~100%) and their respective performance baselines.
• Assuming standard 64-frame evaluation applies to streaming benchmarks, which use variable frame counts (e.g., 248+8, 504+8).
• Overlooking that average scores are computed across heterogeneous benchmarks with different evaluation protocols (exact match, LLM-judge, etc.).

Evidence (verbatim from paper)

We evaluate on a wide range of video benchmarks, including Opencompass video leaderboard: VideoMME, Tempcompass, MVBench, MMBench-Video, MLVU, LongVideoBench, and other commonly used video benchmarks: MovieChat1K, CG-Bench, EgoSchema, TemporalBench, Activitynet-qa, LVBench and WorldSense. We use VLMEvalKit and lmms-eval for evaluation.

Citation

@misc{wang2026pointslong,
  title={POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs},
  author={Haicheng Wang et al.},
  year={2026},
  note={arXiv:2604.11627}
}

• arXiv: 2604.11627