# Dexcanvas Success Eval

> This evaluation probes a robot policy's ability to successfully reproduce human-demonstrated dexterous manipulation trajectories in physics simulation. It measures robustness by testing performance under nominal conditions and under controlled initial pose perturbations. Use when the user wants to benchmark on DexCanvas, or asks about evaluating this task. Reports success rate.

- Skill: `qhjqhj00/dexcanvas-success-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/dexcanvas-success-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/dexcanvas-success-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/dexcanvas-success-eval

---


# dexcanvas-success-eval

> DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation — Xu et al. (2025) (arXiv:2510.15786, 2025)

## What this evaluates

This evaluation probes a robot policy's ability to successfully reproduce human-demonstrated dexterous manipulation trajectories in physics simulation. It measures robustness by testing performance under nominal conditions and under controlled initial pose perturbations.

## Datasets

- **DexCanvas** — total 7000; splits: test (32); repo https://github.com/dexrobot/dexcanvas

## Metrics

- `success rate` **(primary)** — range: percent
  - Calculated as the number of successful rollouts divided by the total number of rollouts (100 per policy), expressed as a percentage. A rollout is successful if the policy completes the demonstration trajectory without triggering termination.

## Input / output format

**Input**: Object-manipulation pair configuration, initial object pose (nominal or perturbed by up to 20% of object size in length/width), and trained policy parameters.

**Output**: Boolean success/failure flag per rollout, determined by whether the positional error exceeds 5cm during execution.

## Scoring recipe

```python
success_count = 0
total_rollouts = 0
for policy in policies:
    for _ in range(100):
        trajectory = run_simulation(policy, initial_pose)
        total_rollouts += 1
        if max_positional_error(trajectory) <= 5.0:
            success_count += 1
success_rate = (success_count / total_rollouts) * 100
```

## Common pitfalls

- Termination is triggered specifically when positional error exceeds 5cm, not by time limits or other constraints.
- Perturbations are defined as a 20% shift in object size along length and width axes, not uniform random noise.
- Success rate is averaged across 32 representative object-manipulation pairs, not reported per-pair in the headline metric.

## Evidence (verbatim from paper)

> Each policy was rolled out 100 times in simulation, with success defined as completing the demonstration trajectory without termination (triggered when positional error exceeded 5cm). Policies achieved an 80.15% success rate under nominal conditions, demonstrating effective reproduction of human demonstrations in physics simulation. When initial object poses were perturbed by up to 20% of object size, the success rate decreased moderately to 62.54%—a drop of only 17.61 percentage points.

## Citation

```bibtex
@misc{xu2025dexcanvas,
  title={DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation},
  author={Xu et al. (2025)},
  year={2025},
  note={arXiv:2510.15786}
}
```

- arXiv: 2510.15786