Manipulation Transfer Eval

manipulation-transfer-eval

Learning Transferable Motor Skills with Hierarchical Latent Mixture Policies — Rao et al. (2021) (arXiv:2112.05062, 2021)

What this evaluates

Evaluates whether learned hierarchical motor skills can transfer across different object geometries, downstream stacking tasks, and observation modalities (state vs. vision). It probes sample efficiency, directed exploration, and performance under varying reward sparsities (dense, staged sparse, fully sparse).

Datasets

• red_on_blue_stacking — total ?; splits: (unstated)
• all_pairs_stacking — total ?; splits: (unstated)

Metrics

• reward (primary) — range: other
  • Mean cumulative reward collected over evaluation episodes. Computed separately for dense, staged sparse, and fully sparse reward settings to measure task success and sample efficiency.
• state_coverage_variance — range: other
  • Variance of state subsets (joints, grasp sensor, object poses) over a single episode, averaged across episodes. Used to quantify the diversity of zero-shot exploration.

Input / output format

Input: Environment state observations including proprioception (joint angles/velocities), object poses, and optionally RGB images for vision-based policies.

Output: Continuous action vector controlling the Sawyer arm's joints and gripper.

Scoring recipe

def compute_metrics(rollouts):
    rewards = [sum(r for r in episode) for episode in rollouts]
    avg_reward = sum(rewards) / len(rewards)
    coverage = []
    for episode in rollouts:
        states = [obs['state'] for obs in episode]
        coverage.append(np.var(states, axis=0))
    avg_coverage = np.mean(coverage, axis=0)
    return avg_reward, avg_coverage

Common pitfalls

• Offline datasets are collected from a single source task but evaluated on geometrically distinct object sets and compositional tasks, which can mask adaptation failures.
• Vision-based transfer experiments still rely on privileged object states during training for KL-regularization, so zero-shot vision-only evaluation is not reported.
• Performance metrics vary significantly across reward sparsities; comparing dense vs. fully sparse results without noting the reward structure leads to misleading conclusions about sample efficiency.

Evidence (verbatim from paper)

We also record the total reward (dense and sparse staged). The results are reported in Table 1. While all approaches achieve some zero-shot dense reward (with BC the most effective), HeLMS receives a sparse staged reward an order of magnitude greater.

Citation

@misc{rao2021learning,
  title={Learning Transferable Motor Skills with Hierarchical Latent Mixture Policies},
  author={Rao et al. (2021)},
  year={2021},
  note={arXiv:2112.05062}
}

• arXiv: 2112.05062