# Being H05 Robot Eval

> Evaluates cross-embodiment generalization and manipulation capabilities of Vision-Language-Action models across heterogeneous real robots and simulation benchmarks. Probes spatial reasoning, long-horizon planning, bimanual coordination, and zero-shot transfer to unseen task-embodiment pairs. Use when the user wants to benchmark on Real-robot task suite, LIBERO, RoboCasa, or asks about evaluating this task. Reports success rate (%).

- Skill: `qhjqhj00/being-h05-robot-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/being-h05-robot-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/being-h05-robot-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/being-h05-robot-eval

---


# being-h05-robot-eval

> Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization — Hao Luo et al. (2026) (arXiv:2601.12993, 2026)

## What this evaluates

Evaluates cross-embodiment generalization and manipulation capabilities of Vision-Language-Action models across heterogeneous real robots and simulation benchmarks. Probes spatial reasoning, long-horizon planning, bimanual coordination, and zero-shot transfer to unseen task-embodiment pairs.

## Datasets

- **Real-robot task suite** — total ?; splits: test (-1)
- **LIBERO** — total ?; splits: test (-1)
- **RoboCasa** — total ?; splits: test (-1)

## Metrics

- `success rate (%)` **(primary)** — range: percent
  - Binary success/failure per trial based on pre-defined objective criteria. Reported as the mean success rate over K trials (K=20 for real robots, 500 per suite for LIBERO, 50 per task for RoboCasa).

## Input / output format

**Input**: Multi-view RGB images (224x224 resolution), natural language task instructions, and robot proprioceptive/sensory observations.

**Output**: Unified action space tokens representing robot motor commands (e.g., joint velocities/positions, gripper states).

## Scoring recipe

```python
def compute_success_rate(model, tasks, trials_per_task=20):
    total_successes = 0
    total_trials = 0
    for task in tasks:
        for config in task.predefined_layouts:
            for trial in range(trials_per_task):
                obs = get_observation(config, trial)
                action = model.predict(obs)
                success = check_objective_criteria(action, config)
                total_successes += success
                total_trials += 1
    return (total_successes / total_trials) * 100
```

## Common pitfalls

- Operator bias can skew real-robot results; the paper enforces a blind black-box inference server to prevent this.
- Success criteria must be strictly pre-defined and objective, as human operators record binary outcomes without knowing the model identity.
- LIBERO and RoboCasa use different trial counts (500 vs 50) and scene splits, so averaging must follow each benchmark's specific protocol.

## Evidence (verbatim from paper)

> We report the mean success rate over 50 evaluation episodes per task. Being-H0.5(specialist) reaches a 98.9% average success rate on LIBERO, with strong performance across all suites including LIBERO-Long (97.4%).

## Citation

```bibtex
@misc{luo2026beingh05,
  title={Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization},
  author={Hao Luo et al. (2026)},
  year={2026},
  note={arXiv:2601.12993}
}
```

- arXiv: 2601.12993