Omni Safetybench Eval

omni-safetybench-eval

Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models — Pan et al. (2025) (arXiv:2508.07173, 2025)

What this evaluates

Evaluates the safety alignment and refusal capabilities of Audio-Visual Large Language Models (OLLMs) when exposed to harmful unimodal, dual-modal, and omni-modal inputs. It specifically probes whether models maintain consistent safety boundaries across modality combinations and reveals vulnerabilities in cross-modal comprehension-aware safety.

Datasets

• Omni-SafetyBench — total 972; splits: test (972); repo https://github.com/THU-BPM/Omni-SafetyBench

Metrics

• C-ASR — range: percent
  • Conditional Attack Success Rate: the percentage of harmful inputs where the model fails to refuse or generates unsafe content, determined by judge model evaluation.
• C-RR — range: percent
  • Conditional Refusal Rate: the percentage of harmful inputs where the model explicitly refuses to answer, determined by judge model evaluation.
• Safety-score (primary) — range: [0, 1]
  • A comprehensive metric combining understanding, safety, and refusal evaluations by the Qwen-Plus judge model to quantify overall safety performance on a 0-1 scale.
• CMSC-score — range: [0, 1]
  • Cross-Modal Safety Consistency score: quantifies how consistently a model maintains its safety performance across different modality paradigms (unimodal, dual-modal, omni-modal).

Input / output format

Input: Audio, visual, and text prompts (unimodal, dual-modal, or omni-modal) containing harmful or joint-harm cases.

Output: Model-generated response (text/audio/video), which is subsequently evaluated by the Qwen-Plus judge model for understanding, safety, and refusal.

Scoring recipe

judge = Qwen_Plus()
results = []
for instance in dataset:
    response = model.generate(instance.input)
    judgment = judge.evaluate(response, instance.prompt)
    results.append(judgment)

c_asr = sum(1 for r in results if r.safety == unsafe) / len(results)
c_rr = sum(1 for r in results if r.refusal == true) / len(results)
safety_score = mean(r.safety_score for r in results)
cmsc_score = compute_consistency_across_modalities(results)

Common pitfalls

• Assuming unimodal safety performance generalizes to dual-modal or omni-modal inputs, as the benchmark shows significant performance drops under complex joint inputs.
• Relying solely on automated refusal detection without comprehension-aware judging, which can miss cases where models understand the harm but fail to refuse appropriately.
• Ignoring cross-modal consistency; a model might score well on average but fail catastrophically on specific modality combinations (e.g., image-audio-text).

Evidence (verbatim from paper)

We employ the Conditional Attack Success Rate (C-ASR), Conditional Refusal Rate (C-RR), and a comprehensive metric, the Safety-score, to represent safety performance. To evaluate cross-modal safety consistency, we use the Cross-Modal Safety Consistency score (CMSC-score) as a quantification metric. To balance labeling accuracy and cost, we use Qwen-Plus... as the judge model to evaluate responses for understanding, safety, and refusal

Citation

@misc{pan2025omisafetybench,
  title={Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models},
  author={Pan et al. (2025)},
  year={2025},
  note={arXiv:2508.07173}
}

• arXiv: 2508.07173