Hoivg Bench Eval

hoivg-bench-eval

OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation — Zhou et al. (2026) (arXiv:2604.11804, 2026)

What this evaluates

Evaluates a video generation model's ability to synthesize high-fidelity videos conditioned on multimodal inputs (reference images, audio, pose, and text) while maintaining reference consistency, audio-visual synchronization, and temporal coherence.

Datasets

• HOIVG-Bench — total ?; splits: test (-1)
• EMTD — total ?; splits: test (-1)

Metrics

• NexusScore (primary) — range: [0, 1]
  • Composite metric evaluating overall video quality, reference preservation, and condition adherence across multimodal settings.
• FaceSim — range: [0, 1]
  • Measures facial similarity and identity preservation between the generated video frames and the input reference image.
• TA — range: [0, 1]
  • Text Alignment score measuring the degree to which the generated video adheres to the provided text prompt.
• Sync-C — range: [0, 1]
  • Audio-visual synchronization consistency score evaluating temporal alignment between audio cues and visual actions.
• Sync-D — range: [0, 1]
  • Audio-visual synchronization detail score measuring fine-grained frame-wise interaction and temporal coherence.

Input / output format

Input: Text prompt, reference image(s), audio clip, and pose sequence (optional depending on setting: R2V, RA2V, RP2V, RAP2V)

Output: Video sequence (480p or 720p)

Scoring recipe

def evaluate(predictions, gold):
    metrics = {}
    metrics['FaceSim'] = compute_face_similarity(predictions, gold['ref_img'])
    metrics['NexusScore'] = compute_nexus_score(predictions, gold['text'], gold['ref_img'])
    metrics['TA'] = compute_text_alignment(predictions, gold['text'])
    metrics['Sync-C'] = compute_sync_consistency(predictions, gold['audio'])
    metrics['Sync-D'] = compute_sync_detail(predictions, gold['audio'])
    return metrics

Common pitfalls

• Baselines often lack support for the full set of four multimodal conditions, requiring evaluation across different input subsets (R2V, RA2V, RP2V) rather than a single unified setting.
• Objective metrics like FaceSim may drop in pose-conditioned settings (RP2V) due to viewpoint shifts and facial morphology changes, which does not necessarily indicate poor generation quality.
• Human evaluation is conducted on small subsets (20 samples) with limited participants (30-33), which may not fully capture temporal dynamics or perceived realism compared to frame-level metrics.

Evidence (verbatim from paper)

In the R2V setting, our method matches the reference preservation capabilities of specialized methods like Phantom-14B, as evidenced by our comparable FaceSim and NexusScore. Notably, OmniShow exhibits distinct advantages as a unified framework in more complex scenarios. In the RA2V setting, while dedicated baselines like HuMo-17B may show slight gains in TA, our approach delivers leading performance in other metrics like NexusScore and Sync-C.

Citation

@misc{zhou2026omnishow,
  title={OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation},
  author={Zhou et al. (2026)},
  year={2026},
  note={arXiv:2604.11804}
}

• arXiv: 2604.11804