mujoco
MuJoCo is a physics engine for robotics: a model (MJCF XML → MjModel), a state (MjData), a step
(mj_step). Tools: $MUJOCO_PYTHON (the pinned venv), $MENAGERIE (the robots), harness_mujoco
on PYTHONPATH (load, servos, record, PD hold). Never install another MuJoCo.
Simulate, record — the pane runs it live
"$MUJOCO_PYTHON" sim/hello.py # → out/rollout.qpos.json (+ .model.xml, rollout.json, rollout.mp4)
"$MUJOCO_PYTHON" "$MUJOCO_TOOLCHAIN/verdict.py" # record() already refreshes it; run it after anything else
from harness_mujoco import load_menagerie, load_xml, pd_hold, record
model, data = load_menagerie("unitree_go2", servos=(60, 2)) # torque motors → position servos (PD in the model)
def ctrl(model, data, t): # called before every step
data.ctrl[:] = model.key_ctrl[0] # joint-angle targets: hold the "home" pose
record(model, data, ctrl, seconds=4, track="base") # track: the body the cameras follow
The pane is MuJoCo's simulate, not a video player. It opens out/rollout.qpos.json and runs
the model live in the browser (MuJoCo's WebAssembly build): from the rollout's first frame, with
the controls your controller produced, so the robot does what it did — and then the user can shove
it, drive any actuator from a slider, pose joints, load keyframes, look at contacts and forces, or
replay the exact recording frame by frame. What record writes is what makes that work:
rollout.qpos.json — per frame: time, qpos, qvel, ctrl (and act). It is written while the
rollout runs ("status": "recording"), so the pane shows the run in progress.rollout.model.xml — when you built or edited the model with MjSpec (servos, added bodies,
swapped actuators), the compiled model as MJCF, so the pane simulates your model and not the file
you started from. Runtime edits (model.opt.timestep = …, model.dof_damping[:] = …) ride in the
rollout as model_patch. All automatic.rollout.json — the report the verdict reads. rollout.mp4 — a video to share; video=False
skips it while you iterate (faster). The pane never opens on the video.
Put the low-level controller in the model. The pane re-simulates with the recorded ctrl. A
position servo's target is a pose, and replaying poses keeps the robot standing and reacting to
pushes after the recording ends. Torques computed in Python (pd_hold on a motor, an MPC, a raw
torque policy) replay open-loop and drift. So for legged robots: servos=(kp, kv) (Go2/A1: 60, 2
to start) and command joint angles; the G1 already has position actuators. Keep keyframes in the
MJCF (<key name qpos ctrl>) for poses worth loading in the pane.
The rollout must know which MJCF it came from. load_xml, load_menagerie and a plain
MjModel.from_xml_path or MjSpec.from_file(...).compile() are traced automatically; a model built
from a string needs record(..., model_path="scenes/mine.xml") — save it under scenes/ first — or
the pane has nothing to run and the verdict says so. Keep rollouts short while iterating (2–4 s).
The robots (Menagerie, pinned)
unitree_go2 (quadruped, 12 torque motors → use servos, keyframe "home"), unitree_go1,
unitree_a1, unitree_g1 (humanoid, 29 DoF, position actuators, keyframe "stand"), unitree_h1
(humanoid), berkeley_humanoid, booster_t1. Each has scene.xml (robot + floor + light) and
<robot>.xml; the MJX variants (scene_mjx.xml) are the ones to train with. Body and joint names:
[model.body(i).name for i in range(model.nbody)].
MJCF, the parts that matter
<worldbody> → nested <body pos quat> with <joint type="hinge|slide|ball|free" axis range damping>
and <geom type="box|sphere|capsule|cylinder|mesh|plane" size mass rgba>; <light>, <camera name>
(every named camera is a view in the pane).<actuator>: <motor joint gear ctrlrange> (torque), <position joint kp kv ctrlrange> (servo), <velocity>.
Give actuators names — the pane labels its sliders with them.<sensor>: jointpos, framepos, accelerometer, touch… — the pane lists and plots them live.<option timestep="0.002" gravity>; <keyframe><key name qpos ctrl/> for start poses.<default class> and <include file> keep a robot's XML short; <asset><mesh file> for STL/OBJ.- Contacts:
<geom condim friction>; <contact><exclude> for self-collisions that should not happen.
Controllers and policies
- Servos:
data.ctrl[:] = q_target (joint angles). Torque motors: data.ctrl[:] = tau, or pd_hold(kp, kd).
- Gaits by hand: a phase
t * 2π * f per leg, targets from the home pose plus sinusoids; keep it slow.
- A trained policy: load weights (
.npz, .pt), map data.qpos/qvel/sensordata → observation → action → data.ctrl
(position targets, so the pane can re-simulate it).
- Training:
toolchain/install-training.sh adds JAX, MJX and MuJoCo Playground
(from mujoco_playground import registry; env = registry.load("Go2JoystickFlatTerrain"); PPO via
Brax in mujoco_playground examples). On a Mac JAX runs on the CPU — a smoke run, not a policy; say
so, and point at a GPU machine in Harness's Machines menu for the real run. Save checkpoints under
out/, and record the policy's rollout with record so the pane shows what it learned.
Rules
sim/ holds scripts, scenes/ your MJCF, out/ rollouts; never write inside $MENAGERIE.- Always let
record write the rollout; never hand-roll rollout.qpos.json or point anything at the mp4.
- Divergence (NaN) means the timestep is too large for the gains, or a joint has no range/damping.
- Every request that says "make it walk / stand / reach" is a controller first and a policy second.
1---2name: mujoco3description: Simulate robots and scenes with MuJoCo — Menagerie robots (Unitree Go2, G1, H1, Berkeley Humanoid, Booster T1), your own MJCF, controllers and policies — record rollouts the pane runs as a live simulation, and train with MJX and MuJoCo Playground. Use for any request that ends in a simulation, a rollout or a policy.4---56# mujoco78MuJoCo is a physics engine for robotics: a model (MJCF XML → `MjModel`), a state (`MjData`), a step9(`mj_step`). Tools: `$MUJOCO_PYTHON` (the pinned venv), `$MENAGERIE` (the robots), `harness_mujoco`10on `PYTHONPATH` (load, servos, record, PD hold). Never install another MuJoCo.1112## Simulate, record — the pane runs it live1314```bash15"$MUJOCO_PYTHON" sim/hello.py # → out/rollout.qpos.json (+ .model.xml, rollout.json, rollout.mp4)16"$MUJOCO_PYTHON" "$MUJOCO_TOOLCHAIN/verdict.py" # record() already refreshes it; run it after anything else17```1819```python20from harness_mujoco import load_menagerie, load_xml, pd_hold, record21model, data = load_menagerie("unitree_go2", servos=(60, 2)) # torque motors → position servos (PD in the model)22def ctrl(model, data, t): # called before every step23 data.ctrl[:] = model.key_ctrl[0] # joint-angle targets: hold the "home" pose24record(model, data, ctrl, seconds=4, track="base") # track: the body the cameras follow25```2627**The pane is MuJoCo's `simulate`, not a video player.** It opens `out/rollout.qpos.json` and runs28the model *live* in the browser (MuJoCo's WebAssembly build): from the rollout's first frame, with29the controls your controller produced, so the robot does what it did — and then the user can shove30it, drive any actuator from a slider, pose joints, load keyframes, look at contacts and forces, or31replay the exact recording frame by frame. What `record` writes is what makes that work:3233- `rollout.qpos.json` — per frame: `time`, `qpos`, `qvel`, `ctrl` (and `act`). It is written while the34 rollout runs (`"status": "recording"`), so the pane shows the run in progress.35- `rollout.model.xml` — when you built or edited the model with `MjSpec` (servos, added bodies,36 swapped actuators), the compiled model as MJCF, so the pane simulates *your* model and not the file37 you started from. Runtime edits (`model.opt.timestep = …`, `model.dof_damping[:] = …`) ride in the38 rollout as `model_patch`. All automatic.39- `rollout.json` — the report the verdict reads. `rollout.mp4` — a video to share; `video=False`40 skips it while you iterate (faster). The pane never opens on the video.4142**Put the low-level controller in the model.** The pane re-simulates with the recorded `ctrl`. A43position servo's target is a pose, and replaying poses keeps the robot standing and reacting to44pushes after the recording ends. Torques computed in Python (`pd_hold` on a motor, an MPC, a raw45torque policy) replay open-loop and drift. So for legged robots: `servos=(kp, kv)` (Go2/A1: 60, 246to start) and command joint angles; the G1 already has position actuators. Keep keyframes in the47MJCF (`<key name qpos ctrl>`) for poses worth loading in the pane.4849The rollout must know which MJCF it came from. `load_xml`, `load_menagerie` and a plain50`MjModel.from_xml_path` or `MjSpec.from_file(...).compile()` are traced automatically; a model built51from a string needs `record(..., model_path="scenes/mine.xml")` — save it under `scenes/` first — or52the pane has nothing to run and the verdict says so. Keep rollouts short while iterating (2–4 s).5354## The robots (Menagerie, pinned)5556`unitree_go2` (quadruped, 12 torque motors → use `servos`, keyframe "home"), `unitree_go1`,57`unitree_a1`, `unitree_g1` (humanoid, 29 DoF, position actuators, keyframe "stand"), `unitree_h1`58(humanoid), `berkeley_humanoid`, `booster_t1`. Each has `scene.xml` (robot + floor + light) and59`<robot>.xml`; the MJX variants (`scene_mjx.xml`) are the ones to train with. Body and joint names:60`[model.body(i).name for i in range(model.nbody)]`.6162## MJCF, the parts that matter6364- `<worldbody>` → nested `<body pos quat>` with `<joint type="hinge|slide|ball|free" axis range damping>`65 and `<geom type="box|sphere|capsule|cylinder|mesh|plane" size mass rgba>`; `<light>`, `<camera name>`66 (every named camera is a view in the pane).67- `<actuator>`: `<motor joint gear ctrlrange>` (torque), `<position joint kp kv ctrlrange>` (servo), `<velocity>`.68 Give actuators names — the pane labels its sliders with them.69- `<sensor>`: `jointpos`, `framepos`, `accelerometer`, `touch`… — the pane lists and plots them live.70- `<option timestep="0.002" gravity>`; `<keyframe><key name qpos ctrl/>` for start poses.71- `<default class>` and `<include file>` keep a robot's XML short; `<asset><mesh file>` for STL/OBJ.72- Contacts: `<geom condim friction>`; `<contact><exclude>` for self-collisions that should not happen.7374## Controllers and policies7576- Servos: `data.ctrl[:] = q_target` (joint angles). Torque motors: `data.ctrl[:] = tau`, or `pd_hold(kp, kd)`.77- Gaits by hand: a phase `t * 2π * f` per leg, targets from the home pose plus sinusoids; keep it slow.78- A trained policy: load weights (`.npz`, `.pt`), map `data.qpos/qvel/sensordata` → observation → action → `data.ctrl`79 (position targets, so the pane can re-simulate it).80- **Training**: `toolchain/install-training.sh` adds JAX, MJX and MuJoCo Playground81 (`from mujoco_playground import registry; env = registry.load("Go2JoystickFlatTerrain")`; PPO via82 Brax in `mujoco_playground` examples). On a Mac JAX runs on the CPU — a smoke run, not a policy; say83 so, and point at a GPU machine in Harness's Machines menu for the real run. Save checkpoints under84 `out/`, and record the policy's rollout with `record` so the pane shows what it learned.8586## Rules8788- `sim/` holds scripts, `scenes/` your MJCF, `out/` rollouts; never write inside `$MENAGERIE`.89- Always let `record` write the rollout; never hand-roll `rollout.qpos.json` or point anything at the mp4.90- Divergence (NaN) means the timestep is too large for the gains, or a joint has no range/damping.91- Every request that says "make it walk / stand / reach" is a controller first and a policy second.
autonomous-ai