Multicom Bench Eval

multicom-bench-eval

Skywork UniPic 3.0: Unified Multi-Image Composition via Sequence Modeling — Wei et al. (2026) (arXiv:2601.15664, 2026)

What this evaluates

Evaluates a model's ability to compose multiple source images into a single coherent output while following textual instructions, maintaining image quality, and preserving facial consistency in human-object interaction scenarios.

Datasets

• MultiCom-Bench — total 200; splits: test (200)

Metrics

• VIEScore (primary) — range: [0, 1]
  • Composite score derived from VIEScore evaluation templates, measuring adherence to composition instructions, image quality, and facial consistency. Scores are averaged across dimensions and normalized to [0, 1].

Input / output format

Input: A textual composition instruction/prompt and 2 to 6 source images.

Output: A single generated image.

Scoring recipe

def compute_viescore(predictions, prompts, gold_images):
    total_score = 0.0
    for pred, prompt, gold in zip(predictions, prompts, gold_images):
        s_adherence = viescore_template_adherence(pred, prompt)
        s_quality = viescore_template_quality(pred)
        s_consistency = viescore_template_consistency(pred, gold)
        total_score += mean([s_adherence, s_quality, s_consistency])
    return total_score / len(predictions)

Common pitfalls

• Model performance degrades significantly when extending to 4–6 input images due to sequence modeling limits.
• Evaluation relies on VIEScore AI templates that assess multiple dimensions, requiring careful prompt engineering to avoid scoring bias.
• The benchmark is specifically curated for HOI scenarios, so results may not generalize to non-human-object compositions.

Evidence (verbatim from paper)

Recognizing the absence of standardized evaluation protocols for multi-image composition, we construct MultiCom-Bench, a carefully curated benchmark comprising 200 high-quality triplets specifically targeting HOI scenarios. Following VIEScore, we have designed stable and effective evaluation templates that assess model-generated results in multiple dimensions, including the adherence to composition instructions, image quality, and facial consistency.

Citation

@misc{wei2026skyworkunipic3,
  title={Skywork UniPic 3.0: Unified Multi-Image Composition via Sequence Modeling},
  author={Wei et al. (2026)},
  year={2026},
  note={arXiv:2601.15664}
}

• arXiv: 2601.15664