Vc Ifeval Eval

vc-ifeval-eval

Empowering Reliable Visual-Centric Instruction Following in MLLMs — He et al. (2026) (arXiv:2601.03198, 2026)

What this evaluates

Evaluates multimodal large language models' ability to follow instructions containing vision-dependent constraints, such as spatial, stylistic, and structural requirements. It isolates the contribution of visual input to instruction adherence and assesses generalization on standard visual reasoning tasks.

Datasets

• VC-IFEval — total ?; splits: test (-1)
• MM-IFEval — total ?; splits: test (-1)
• IFEval — total ?; splits: test (-1)

Metrics

• instruction-following accuracy (primary) — range: [0, 1]
  • Standard benchmark-specific accuracy metrics for instruction following and visual reasoning tasks as implemented in VLMEvalKit.

Input / output format

Input: Multimodal inputs (image + text instruction) for VC-IFEval, MM-IFEval, and VQA benchmarks; text-only prompts for IFEval.

Output: Natural language text responses generated by the MLLM, evaluated against ground-truth constraints or answers.

Scoring recipe

For each instance:
  pred = model.generate(image, prompt)
  score = benchmark_metric.evaluate(pred, gold)
aggregate_scores = mean(score across all instances)
return aggregate_scores

Common pitfalls

• Multimodal instruction following benchmarks often conflate language-only capabilities with true visual grounding.
• Isolating visual input contributions requires a hybrid evaluation protocol to prevent language priors from dominating.

Evidence (verbatim from paper)

We evaluate our models on a series of instruction-following and visual reasoning benchmarks, including MM-IFEval (Ding et al., 2025), VC-IFEval, and IFEval (Zhou et al., 2023), where IFEval is language-only while the others are multimodal.

Citation

@misc{he2026empowering,
  title={Empowering Reliable Visual-Centric Instruction Following in MLLMs},
  author={He et al. (2026)},
  year={2026},
  note={arXiv:2601.03198}
}

• arXiv: 2601.03198