fowm-eval
Finetuning Offline World Models in the Real World — Feng et al. (2023) (arXiv:2310.16029, 2023)
What this evaluates
Evaluates offline-to-online finetuning of model-based reinforcement learning world models on continuous control and visuomotor tasks. It probes the model's ability to adapt to seen and unseen task variations with limited online interactions while mitigating extrapolation errors via uncertainty regularization.
Datasets
- D4RL — total ?; splits: Hopper (medium) (-1), Hopper (medium-replay) (-1), AntMaze (medium-play) (-1), AntMaze (medium-diverse) (-1)
- xArm — total ?; splits: push (-1), pick (-1)
- Quadruped Locomotion — total ?; splits: Walk (-1)
- Real xArm — total ?; splits: reach (-1), pick (-1), kitchen (-1)
Metrics
Success rate (%) (primary) — range: percent
- Percentage of trials where the robot successfully completes the task goal (e.g., reaching a target, lifting an object above a threshold, or placing a pot in a sink).
Normalized return — range: other
- D4RL-style normalized cumulative reward, scaled to a [0, 100] range based on random and expert baselines, or raw cumulative reward for quadruped tasks.
Input / output format
Input: 224x224 RGB images from a static third-person camera, robot proprioceptive state, and task goal/target coordinates.
Output: Positional control commands for the robot's end-effector.
Scoring recipe
def compute_success_rate(success_flags, num_trials):
return sum(success_flags) / num_trials * 100
def compute_normalized_return(reward, env_min, env_max):
return (reward - env_min) / (env_max - env_min) * 100
Common pitfalls
- Confusing 'medium' and 'medium-replay' dataset variations for xArm tasks, which have different data generation sources.
- Real-world tasks use manually designed detectors for success and sparse, noisy rewards, which differs from the dense rewards in simulation.
- Failing to report both zero-shot (offline) and post-finetuning (online) performance, as the paper emphasizes the offline-to-online transition.
Evidence (verbatim from paper)
Success rate (xArm) and normalized return (D4RL and quadruped) of methods before and after online finetuning. See Appendix B for task explanations. Mean of 5 seeds.
Citation
@misc{feng2023fowm,
title={Finetuning Offline World Models in the Real World},
author={Feng et al. (2023)},
year={2023},
note={arXiv:2310.16029}
}
1---2name: fowm-eval3description: Evaluates offline-to-online finetuning of model-based reinforcement learning world models on continuous control and visuomotor tasks. It probes the model's ability to adapt to seen and unseen task variations with limited online interactions while mitigating extrapolation errors via uncertainty regularization. Use when the user wants to benchmark on D4RL, xArm, Quadruped Locomotion, Real xArm, or asks about evaluating this task. Reports Success rate (%).4---56# fowm-eval78> Finetuning Offline World Models in the Real World — Feng et al. (2023) (arXiv:2310.16029, 2023)910## What this evaluates1112Evaluates offline-to-online finetuning of model-based reinforcement learning world models on continuous control and visuomotor tasks. It probes the model's ability to adapt to seen and unseen task variations with limited online interactions while mitigating extrapolation errors via uncertainty regularization.1314## Datasets1516- **D4RL** — total ?; splits: Hopper (medium) (-1), Hopper (medium-replay) (-1), AntMaze (medium-play) (-1), AntMaze (medium-diverse) (-1)17- **xArm** — total ?; splits: push (-1), pick (-1)18- **Quadruped Locomotion** — total ?; splits: Walk (-1)19- **Real xArm** — total ?; splits: reach (-1), pick (-1), kitchen (-1)2021## Metrics2223- `Success rate (%)` **(primary)** — range: percent24 - Percentage of trials where the robot successfully completes the task goal (e.g., reaching a target, lifting an object above a threshold, or placing a pot in a sink).25- `Normalized return` — range: other26 - D4RL-style normalized cumulative reward, scaled to a [0, 100] range based on random and expert baselines, or raw cumulative reward for quadruped tasks.2728## Input / output format2930**Input**: 224x224 RGB images from a static third-person camera, robot proprioceptive state, and task goal/target coordinates.3132**Output**: Positional control commands for the robot's end-effector.3334## Scoring recipe3536```python37def compute_success_rate(success_flags, num_trials):38 return sum(success_flags) / num_trials * 1003940def compute_normalized_return(reward, env_min, env_max):41 return (reward - env_min) / (env_max - env_min) * 10042```4344## Common pitfalls4546- Confusing 'medium' and 'medium-replay' dataset variations for xArm tasks, which have different data generation sources.47- Real-world tasks use manually designed detectors for success and sparse, noisy rewards, which differs from the dense rewards in simulation.48- Failing to report both zero-shot (offline) and post-finetuning (online) performance, as the paper emphasizes the offline-to-online transition.4950## Evidence (verbatim from paper)5152> Success rate (xArm) and normalized return (D4RL and quadruped) of methods before and after online finetuning. See Appendix B for task explanations. Mean of 5 seeds.5354## Citation5556```bibtex57@misc{feng2023fowm,58 title={Finetuning Offline World Models in the Real World},59 author={Feng et al. (2023)},60 year={2023},61 note={arXiv:2310.16029}62}63```6465- arXiv: 2310.16029
qhjqhj00