# Humanoid Everyday Eval

> Evaluates imitation learning and vision-language-action policies on open-world humanoid manipulation. It probes robustness to high-dimensional action spaces, multimodal sensor fusion, and fine-grained visuospatial perception across locomotion, tool use, and precise manipulation tasks. Use when the user wants to benchmark on Humanoid Everyday, or asks about evaluating this task. Reports success rate.

- Skill: `qhjqhj00/humanoid-everyday-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/humanoid-everyday-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/humanoid-everyday-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/humanoid-everyday-eval

---


# humanoid-everyday-eval

> Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulation — Zhao et al. (2025) (arXiv:2510.08807, 2025)

## What this evaluates

Evaluates imitation learning and vision-language-action policies on open-world humanoid manipulation. It probes robustness to high-dimensional action spaces, multimodal sensor fusion, and fine-grained visuospatial perception across locomotion, tool use, and precise manipulation tasks.

## Datasets

- **Humanoid Everyday** — total 10300; splits: train (-1), test (-1)

## Metrics

- `success rate` **(primary)** — range: percent
  - Calculated as the number of successful task completions divided by the total number of trials (10 per task), multiplied by 100.

## Input / output format

**Input**: Multimodal sensor observations (RGB, depth, LiDAR, tactile, IMU) and natural language task instructions, synchronized at 30Hz.

**Output**: 28-degree-of-freedom (DoF) robot action commands.

## Scoring recipe

```python
def compute_success_rate(successful_trials, total_trials=10):
    return (successful_trials / total_trials) * 100
# Evaluated over 10 trials per task. Average success rate is computed across all 7 tasks.
```

## Common pitfalls

- High-dimensional action space (28 DoFs) causes most end-to-end imitation policies to struggle significantly.
- Downsampling VLA model actions from 30Hz to 2Hz alleviates decoding failures but reduces motion smoothness.
- 3D point cloud inputs become unreliable in locomotion tasks due to large frame-to-frame camera changes.
- Fine-grained visuospatial perception is required for high-precision tasks, which current models largely lack.

## Evidence (verbatim from paper)

> We present the results of these policies in Table II. where each policy is evaluated over 10 trials on seven tasks. TABLE II: Success rates of imitation learning methods on the Humanoid Everyday Dataset.

## Citation

```bibtex
@misc{zhao2025humanoid,
  title={Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulation},
  author={Zhao et al. (2025)},
  year={2025},
  note={arXiv:2510.08807}
}
```

- arXiv: 2510.08807