Valueground Eval

valueground-eval

ValueGround: Evaluating Culture-Conditioned Visual Value Grounding in MLLMs — Wang et al. (2026) (arXiv:2604.06484, 2026)

What this evaluates

Evaluates whether multimodal large language models (MLLMs) can maintain consistent culture-conditioned value judgments when response options are replaced with minimally contrastive visual proxies. It probes cross-modal prediction stability and the ability to ground textual value tendencies in subtle visual contrasts.

Datasets

• ValueGround — total ?; splits: test (-1); repo https://github.com/NL2G/ValueGround

Metrics

• accuracy (primary) — range: percent
  • Percentage of correct predictions where the model's output matches the ground-truth option (A or B) for each country-question-image pair.

Input / output format

Input: Main task: a country $c$, a World Values Survey question $q$, and a pair of minimally contrastive images $I_q=(I_A, I_B)$ representing the two verbal options.

Output: A single choice indicating either option A or option B, corresponding to the image that better matches the value tendency associated with country $c$ for question $q$.

Scoring recipe

correct = 0
total = 0
for country, question, image_pair, gold_option in dataset:
    pred = model(country, question, image_pair)
    if pred == gold_option:
        correct += 1
    total += 1
accuracy = (correct / total) * 100

Common pitfalls

• Models may rely on country-conditioned textual priors rather than actually grounding the judgment in the visual contrast, leading to inflated text-only performance that doesn't transfer to the main task.
• High alignment accuracy (92.8%) does not guarantee main task success, as the bottleneck is multimodal integration under country conditioning, not visual contrast recognition.

Evidence (verbatim from paper)

Averaged across models, accuracy drops from 72.8% in text-only and 92.8% in alignment to 65.8% on the main task. This pattern indicates that success cannot be explained by country-conditioned textual prediction or visual contrast recognition alone.

Citation

@misc{wang2026valueground,
  title={ValueGround: Evaluating Culture-Conditioned Visual Value Grounding in MLLMs},
  author={Wang et al. (2026)},
  year={2026},
  note={arXiv:2604.06484}
}

• arXiv: 2604.06484