qhjqhj00 fysics-eval
Exploring Physical Intelligence Emergence via Omni-Modal Architecture and Physical Data Engine — Minghao Han et al. (2026) (arXiv:2602.07064, 2026)
What this evaluates
Evaluates multimodal large language models' ability to perceive, reason about, and generate physical attributes and laws from images, videos, and audio. It probes causal physical reasoning, material property mapping, and cross-modal consistency rather than superficial pattern matching.
Datasets
- FysicsEval — total ?; splits: test (-1); repo https://github.com/Fysics-AI/FysicsEval
Metrics
average score(primary) — range: percent- Arithmetic mean of accuracy or normalized scores across all benchmark subsets. Reported as a percentage or raw score depending on the subset.
Input / output format
Input: Multimodal inputs (images, videos, or audio) paired with text prompts or questions targeting physical attributes, causal relationships, or material properties.
Output: Text-based answers or generated images/videos, depending on the task modality (perception vs. generation).
Scoring recipe
def compute_average_score(predictions, gold):
correct = sum(1 for p, g in zip(predictions, gold) if p == g)
return (correct / len(gold)) * 100
Common pitfalls
- Models may rely on surface-level visual patterns rather than inferring latent causal physical relationships, leading to poor generalization on unseen physical constraints.
- Optimizing heavily for physical causal chains can interfere with purely abstract mathematical or diagrammatic reasoning, causing performance drops on benchmarks like MathVista.
- Generation models might produce high-fidelity visuals that violate physical laws (e.g., incorrect density or elasticity) if not explicitly constrained during training or evaluation.
Evidence (verbatim from paper)
As presented in Table [1], OmniFysics demonstrates exceptional performance on physical perception and reasoning benchmarks, not only significantly outperforming models of comparable scale but also exhibiting strong competitiveness against larger state-of-the-art systems. Overall, it achieves the best performance with a leading average score of 49.97.
Citation
@misc{han2026exploring,
title={Exploring Physical Intelligence Emergence via Omni-Modal Architecture and Physical Data Engine},
author={Minghao Han et al. (2026)},
year={2026},
note={arXiv:2602.07064}
}
- arXiv: 2602.07064