Paper Rob Robocasa

paper_rob__robocasa

RoboCasa is a large-scale simulation framework for training generalist robots on everyday kitchen tasks. Built on robosuite/MuJoCo, it provides 365 tasks across 2,500+ diverse kitchen scenes with 3,200+ object assets. Supports multiple robot embodiments (mobile manipulators, humanoids, quadrupeds) and integrates with Diffusion Policy, pi0, and GR00T.

Paper Info

Method Overview

RoboCasa is not a learning algorithm — it's a simulation benchmark + data generation platform. Key components:

1. Kitchen scenes — 60 layouts × 60 styles = 2,500+ unique environments with AI-generated textures
2. Task suite — 365 tasks: atomic skills (pick/place, door, drawer, knob, lever, button, insertion, navigation, rack, lid) + LLM-generated composite tasks
3. Object assets — 3,200+ objects across 150+ categories from Objaverse + Lightwheel + AI-generated
4. Demonstration data — 600+ hrs human teleoperation + 1,600+ hrs MimicGen synthetic data
5. Cross-embodiment — PandaOmron (default), humanoids (GR1, G1), quadrupeds (GoogleRobot)
6. Policy integration — Works with robomimic (BC-Transformer), Diffusion Policy, pi0, GR00T

Paper-Code Mapping

Setup

Dependencies

• Python 3.9+
• mujoco==3.3.1
• numpy==2.2.5, scipy==1.15.3, numba==0.61.2
• tianshou==0.4.10, lerobot==0.3.3, gymnasium
• opencv-python, h5py, Pillow, pyyaml, imageio
• Requires robosuite (installed as dependency)

Installation

git clone https://github.com/robocasa/robocasa.git
cd robocasa
pip install -e .

# Download kitchen assets (textures, fixtures, objects)
python robocasa/scripts/download_kitchen_assets.py

# Set dataset path
python robocasa/scripts/setup_macros.py

Asset Downloads

Environment System

Creating an Environment

import robosuite
from robosuite.controllers import load_composite_controller_config

env = robosuite.make(
    env_name="PickPlaceCounterToCabinet",     # Task class name
    robots="PandaOmron",
    controller_configs=load_composite_controller_config(robot="PandaOmron"),
    has_renderer=True,                         # On-screen rendering
    has_offscreen_renderer=True,               # For camera obs
    use_camera_obs=True,
    camera_names=["robot0_agentview_left", "robot0_agentview_right", "robot0_eye_in_hand"],
    camera_heights=256,
    camera_widths=256,
    control_freq=20,                           # 20 Hz control
    horizon=1000,                              # Max steps per episode
    layout_ids=1,                              # Kitchen layout (1-60)
    style_ids=1,                               # Kitchen style (1-60)
    seed=42,
)

Or use the convenience wrapper:

from robocasa.utils.env_utils import create_env

env = create_env(
    env_name="PickPlaceCounterToCabinet",
    robots="PandaOmron",
    split="target",                  # "target", "pretrain", "all"
    obj_instance_split="train",      # "train", "test"
    generative_textures="100p",      # AI textures
)

Kitchen __init__ Key Arguments

Scene Splits

Observation Space

Action Space

Workspace end-effector control (default for PandaOmron):

Task System

Task Hierarchy

Kitchen (base)
├── Atomic tasks (~40)
│   ├── PickPlace → PickPlaceCounterToCabinet, PickPlaceSinkToCounter, ...
│   ├── OpenDoor → OpenSingleDoor, OpenDoubleDoor, ...
│   ├── CloseDrawer, OpenDrawer
│   ├── TwistKnob, TurnOnStove, TurnOffStove
│   ├── TurnLever → TurnOnSinkFaucet, TurnOnFaucetSpout, ...
│   ├── PressButton → TurnOnMicrowave, ArrangeBowlsOnStove, ...
│   ├── Insertion → PrepareCoffeeMug, InsertToRack, ...
│   └── Navigation → NavigateTo, ...
└── Composite tasks (~325)
    ├── Brewing/ → BrewCoffee, MakeTea, ...
    ├── Washing/ → WashDishes, WashVegetables, ...
    ├── Restocking/ → RestockPantry, RestockFridge, ...
    ├── MeatPrep/ → SeasonMeat, MarinateChicken, ...
    └── ... (20 categories total)

Defining a New Task

from robocasa.environments.kitchen.kitchen import *

class MyCustomTask(Kitchen):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    def _setup_kitchen_references(self):
        """Register fixture references for the task."""
        super()._setup_kitchen_references()
        self.counter = self.register_fixture_ref(
            "counter", dict(id=FixtureType.COUNTER)
        )
        self.cabinet = self.register_fixture_ref(
            "cabinet", dict(id=FixtureType.CABINET)
        )
        self.init_robot_base_ref = self.counter  # Robot spawns near counter

    def _get_obj_cfgs(self):
        """Define object placements."""
        return [dict(
            name="obj",
            obj_groups="fruit",           # Sample from fruit category
            graspable=True,
            placement=dict(
                fixture=self.counter,
                size=(0.50, 0.30),         # Sampling region size
                pos=("ref", -1.0),         # Position relative to reference
            ),
        )]

    def _setup_scene(self):
        """Set initial scene state (e.g., open doors)."""
        super()._setup_scene()
        self.cabinet.open_door(env=self)

    def get_ep_meta(self):
        """Return episode metadata including language instruction."""
        ep_meta = super().get_ep_meta()
        obj_lang = self.get_obj_lang()
        ep_meta["lang"] = f"Pick the {obj_lang} and place it in the cabinet."
        return ep_meta

    def _check_success(self):
        """Check if task is completed."""
        import robocasa.utils.object_utils as OU
        return (OU.obj_inside_of(self, "obj", self.cabinet)
                and OU.gripper_obj_far(self))

Tasks are auto-registered via the KitchenEnvMeta metaclass — just define the class and it becomes available via robosuite.make(env_name="MyCustomTask", ...).

Fixture Types

from robocasa.models.fixtures.fixture import FixtureType

# Key types:
FixtureType.COUNTER          # 14
FixtureType.CABINET          # 18
FixtureType.FRIDGE           # 8
FixtureType.SINK             # 4
FixtureType.STOVE            # 2
FixtureType.MICROWAVE        # 1
FixtureType.OVEN             # 3
FixtureType.DISHWASHER       # 9
FixtureType.COFFEE_MACHINE   # 5
FixtureType.DRAWER           # 23
FixtureType.ISLAND           # 15
FixtureType.DISH_RACK        # 26

Demonstration Data

Dataset Registry

from robocasa.utils.dataset_registry import ATOMIC_TASK_DATASETS, COMPOSITE_TASK_DATASETS
from robocasa.utils.dataset_registry_utils import get_ds_path, get_ds_meta

# Get dataset path for a task
path = get_ds_path("PickPlaceCounterToCabinet", source="human", split="pretrain")
meta = get_ds_meta("PickPlaceCounterToCabinet")  # horizon, etc.

HDF5 Dataset Format

dataset.hdf5
├── data/
│   ├── demo_0/
│   │   ├── states         # (T, state_dim) MuJoCo simulation state
│   │   ├── actions        # (T, action_dim) action vectors
│   │   ├── obs/           # Observations dict
│   │   │   ├── robot0_agentview_left_image   # (T, H, W, 3)
│   │   │   ├── robot0_eye_in_hand_image      # (T, H, W, 3)
│   │   │   ├── robot0_eef_pos                # (T, 3)
│   │   │   └── ...
│   │   └── ep_meta        # JSON: {"lang": "...", "object_cfgs": {...}}
│   ├── demo_1/
│   └── ...
└── env_args               # JSON: env creation kwargs

Data Splits

Gym Wrapper

from robocasa.wrappers.gym_wrapper import RoboCasaGymEnv

env = RoboCasaGymEnv(
    env_name="PickPlaceCounterToCabinet",
    camera_names=["robot0_agentview_left", "robot0_agentview_right", "robot0_eye_in_hand"],
    camera_widths=128,
    camera_heights=128,
    split="test",
)

obs, info = env.reset()
obs, reward, terminated, truncated, info = env.step(action)

Supported Robots

Predefined Evaluation Scenes

from robocasa.utils.env_utils import KITCHEN_SCENES_5X5, KITCHEN_SCENES_5X1

# 5×5 = 25 scenes for thorough eval
KITCHEN_SCENES_5X5  # layouts [11,15,18,40,50] × styles [14,28,34,46,58]

# 5×1 = 5 scenes for quick eval
KITCHEN_SCENES_5X1  # layouts [11,15,18,40,50] × style [34]

Repo Structure

Tips & Gotchas

• Run download_kitchen_assets.py before anything else — environments won't render without textures and fixture models.
• Run setup_macros.py to set DATASET_BASE_PATH in macros_private.py — needed for dataset downloading/loading.
• Layout/style IDs start at 1, not 0. Use negative IDs for groups (-1=test, -2=train, -3=all).
• obj_instance_split="test" holds out specific object instances for generalization evaluation — always use this for eval.
• Composite tasks often need "aigen" in obj_registries — some composite tasks auto-add it, but if objects fail to load, add it manually.
• Control frequency is 20 Hz (not 50 Hz like ALOHA). Real robot controller runs at 15 Hz.
• The render_camera arg controls the on-screen view; camera_names controls observation cameras — they're independent.
• MimicGen data only exists for atomic tasks — composite tasks have human demos only.
• Auto-registered tasks: defining a class that inherits from Kitchen automatically registers it via KitchenEnvMeta. No manual registration needed.
• robosuite is the underlying framework — robosuite.make() is the canonical env creation path. RoboCasa extends it with kitchen-specific environments.
• For policy training, use the robomimic integration or the Gym wrapper — don't train directly against the raw robosuite env API.
• Generative textures ("100p") add visual diversity but slow down scene loading. Disable for debugging.