# Hipho Physics Olympiad Eval

> This benchmark evaluates multimodal physical reasoning and advanced problem-solving capabilities on international and regional physics Olympiad exams. It probes a model's ability to interpret complex diagrams, data, and text, perform multi-step logical derivations, and produce accurate solutions under strict, official scoring rubrics. Use when the user wants to benchmark on HiPhO, or asks about evaluating this task. Reports exam score.

- Skill: `qhjqhj00/hipho-physics-olympiad-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/hipho-physics-olympiad-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/hipho-physics-olympiad-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/hipho-physics-olympiad-eval

---


# hipho-physics-olympiad-eval

> PhysicsMinions: Winning Gold Medals in the Latest Physics Olympiads with a Coevolutionary Multimodal Multi-Agent System — Fangchen Yu et al. (2025) (arXiv:2509.24855, 2025)

## What this evaluates

This benchmark evaluates multimodal physical reasoning and advanced problem-solving capabilities on international and regional physics Olympiad exams. It probes a model's ability to interpret complex diagrams, data, and text, perform multi-step logical derivations, and produce accurate solutions under strict, official scoring rubrics.

## Datasets

- **HiPhO** — total ?; splits: test (-1)

## Metrics

- `exam score` **(primary)** — range: other
  - Points awarded per exam based on official marking schemes, evaluated at both answer-level and step-level. Scores vary by exam maximum (e.g., 30 for IPhO, 25 for F=MA).
- `Pass@k` — range: other
  - The maximum exam score achieved across k independent inference attempts for a given problem or exam.

## Input / output format

**Input**: Multimodal physics Olympiad problems containing text, illustrations, variable figures, and data figures.

**Output**: Step-by-step solutions and final answers to the physics problems.

## Scoring recipe

```python
def compute_exam_score(predictions, gold, marking_scheme):
    # Evaluate each prediction using official step-level and answer-level rubrics
    step_points = [score_steps(pred, gold, marking_scheme) for pred in predictions]
    answer_points = [score_final_answer(pred, gold) for pred in predictions]
    # Combine according to official marking scheme
    total_points = [s + a for s, a in zip(step_points, answer_points)]
    # Pass@k takes the highest score across k attempts
    return max(total_points)
```

## Common pitfalls

- Evaluation temperature is fixed at 0.6, not sampled freely.
- Scoring uses both step-level and answer-level breakdowns per official marking schemes, not just exact-match final answers.
- Pass@k reports the best score across k attempts, not the average or median performance.

## Evidence (verbatim from paper)

> Pass@k evaluates a model’s best score over $k$ independent attempts by taking the highest-scoring solution per problem.

## Citation

```bibtex
@misc{yu2025physicsminions,
  title={PhysicsMinions: Winning Gold Medals in the Latest Physics Olympiads with a Coevolutionary Multimodal Multi-Agent System},
  author={Fangchen Yu et al. (2025)},
  year={2025},
  note={arXiv:2509.24855}
}
```

- arXiv: 2509.24855