qhjqhj00 vpt-minecraft-eval
Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos — Bowen Baker et al. (2022) (arXiv:2206.11795, 2022)
What this evaluates
Evaluates an agent's ability to perform complex, multi-step sequential decision-making tasks in a 3D sandbox environment (Minecraft) using a native human-like interface. It probes zero-shot generalization, behavioral cloning fine-tuning, and reinforcement learning fine-tuning for long-horizon crafting and exploration.
Datasets
- webClean — total ?; splits: train (-1)
- contractor_house — total ?; splits: train (-1)
- earlygame_keyword — total ?; splits: train (-1)
Metrics
keypress accuracy— range: percent- Percentage of correctly predicted keypresses from video sequences.
mouse movement R^2— range: [0, 1]- Explained variance of predicted mouse movements compared to ground truth.
reliability(primary) — range: percent- Percentage of evaluation episodes in which the agent successfully obtains a target item (e.g., diamond pickaxe) within the time limit.
crafting/collection rate— range: count/episode- Mean number of specific items collected or crafted per episode, averaged over multiple rollouts.
Input / output format
Input: Per-step visual observation from the Minecraft environment (native human interface at 20Hz).
Output: Discrete keypresses and continuous mouse movements.
Scoring recipe
def evaluate(agent, env, num_episodes=2500, time_limit=600):
success_counts = {item: 0 for item in target_items}
for _ in range(num_episodes):
obs = env.reset()
for t in range(time_limit):
action = agent(obs)
obs, _, done, info = env.step(action)
if done: break
for item in target_items:
if info['collected'][item]:
success_counts[item] += 1
return {item: count / num_episodes for item, count in success_counts.items()}
Common pitfalls
- Catastrophic forgetting during RL fine-tuning can erase previously learned skills if an auxiliary KL divergence loss to the frozen pretrained policy is not applied.
- The native human interface (mouse/keyboard at 20Hz) makes exploration extremely difficult for RL agents compared to simplified action spaces, often causing training to stall without careful initialization.
- Out-of-distribution shift between contractor-labeled data and web-scraped videos can cause validation loss on contractor data to increase during foundation model training, though it does not always correlate with degraded rollout performance.
Evidence (verbatim from paper)
The three-phase training (pretraining, BC fine-tuning, and then RL fine-tuning) succeeds in learning extremely difficult tasks: it achieves over 80% reliability on iron pickaxes, almost 20% reliability on collecting diamonds, and 2.5% reliability on obtaining a diamond pickaxe (Fig. 7d).
Citation
@misc{baker2022vpt,
title={Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos},
author={Bowen Baker et al. (2022)},
year={2022},
note={arXiv:2206.11795}
}
- arXiv: 2206.11795