Training RL Agents
When To Use
Use this skill when a user wants to train, resume, evaluate, or configure reinforcement learning for an Isaac Lab task.
Do not use this skill to design environment observations, rewards, or resets from scratch. Use isaaclab-building-environments (skills/user/create-environments/) for environment construction first, and use isaaclab-debugging-rl-training when training behavior or reward metrics are already failing. Use isaaclab-training-multi-gpu for multi-GPU or multi-node launches and for distributed runs that hang or fail inside NCCL.
Workflow
- Identify the registered task name, workflow type, action space, observation space, target backend, device, and desired RL framework.
- Read the RL training guide and the training tutorial before writing commands or configs.
- Start from an existing agent config under
source/isaaclab_tasks/isaaclab_tasks/ that matches the framework and task family.
- Keep framework-specific config formats separate. Do not mix RSL-RL Python configs with RL-Games, SKRL, or SB3 YAML/config files.
- Use suffixless task names in commands, for example
Isaac-Cartpole.
- From the Isaac Lab checkout, use
uv run isaaclab train and uv run isaaclab play rather than the deprecated per-library scripts under scripts/reinforcement_learning/.
If --rl_library is omitted, the command loads the task's registered default_agent; pass --rl_library to select a different library.
To drive train/play from Python instead, use the train/play functions with TrainingRequest/PlaybackRequest from isaaclab_rl (see Examples).
play applies the environment config's play_mode() method (caps environments at 50 and disables observation noise or corruption, depending on the workflow); override that method on a task config for custom playback.
- Omit visualizer flags for fastest headless training. Use
--viz only when an explicit visualizer is needed.
- Run a small smoke training job before scaling environment count, horizon, network size, or logging integrations.
- For visual observations, confirm the sensor pipeline and renderer requirements before enabling large environment counts.
- For multi-backend tasks, validate training on one backend before introducing backend presets.
- Record the exact task, framework, backend, seed, and config overrides needed to reproduce the result.
Validation
Use this checklist:
- Confirm the task can reset and step without the training runner.
- Confirm the selected agent config belongs to the intended RL framework.
- Run a short training command with a small number of environments.
- Resume or load a checkpoint only after the initial run writes expected artifacts.
- Check logs for device, observation shape, action shape, and backend errors.
For skill changes, run:
uv run --no-project python tools/skills/cli.py check
Maintenance
Keep this skill synchronized with docs/source/concepts/reinforcement_learning.rst, docs/source/isaaclab/installation.rst, RL training tutorials under docs/source/how-to/, and agent configs under source/isaaclab_tasks/isaaclab_tasks/. If framework commands or config formats change, update the official training docs or maintained examples first.
References
1---2name: isaaclab-training-rl-agents3description: Configures and runs Isaac Lab reinforcement learning workflows. Use when selecting RL frameworks, wiring agent configs, launching training, resuming runs, or troubleshooting training setup for Isaac Lab tasks.4audience: user5status: experimental6owners:7 - isaaclab-maintainers8metadata:9 source: https://skillmd.com/skills/isaac-sim/isaaclab-training-rl-agents10---1112# Training RL Agents1314## When To Use1516Use this skill when a user wants to train, resume, evaluate, or configure reinforcement learning for an Isaac Lab task.1718Do not use this skill to design environment observations, rewards, or resets from scratch. Use `isaaclab-building-environments` (`skills/user/create-environments/`) for environment construction first, and use `isaaclab-debugging-rl-training` when training behavior or reward metrics are already failing. Use `isaaclab-training-multi-gpu` for multi-GPU or multi-node launches and for distributed runs that hang or fail inside NCCL.1920## Workflow21221. Identify the registered task name, workflow type, action space, observation space, target backend, device, and desired RL framework.232. Read the RL training guide and the training tutorial before writing commands or configs.243. Start from an existing agent config under `source/isaaclab_tasks/isaaclab_tasks/` that matches the framework and task family.254. Keep framework-specific config formats separate. Do not mix RSL-RL Python configs with RL-Games, SKRL, or SB3 YAML/config files.265. Use suffixless task names in commands, for example `Isaac-Cartpole`.276. From the Isaac Lab checkout, use `uv run isaaclab train` and `uv run isaaclab play` rather than the deprecated per-library scripts under `scripts/reinforcement_learning/`.28 If `--rl_library` is omitted, the command loads the task's registered `default_agent`; pass `--rl_library` to select a different library.29 To drive train/play from Python instead, use the `train`/`play` functions with `TrainingRequest`/`PlaybackRequest` from `isaaclab_rl` (see [Examples](examples.md)).30 `play` applies the environment config's `play_mode()` method (caps environments at 50 and disables observation noise or corruption, depending on the workflow); override that method on a task config for custom playback.317. Omit visualizer flags for fastest headless training. Use `--viz` only when an explicit visualizer is needed.328. Run a small smoke training job before scaling environment count, horizon, network size, or logging integrations.339. For visual observations, confirm the sensor pipeline and renderer requirements before enabling large environment counts.3410. For multi-backend tasks, validate training on one backend before introducing backend presets.3511. Record the exact task, framework, backend, seed, and config overrides needed to reproduce the result.3637## Validation3839Use this checklist:40411. Confirm the task can reset and step without the training runner.422. Confirm the selected agent config belongs to the intended RL framework.433. Run a short training command with a small number of environments.444. Resume or load a checkpoint only after the initial run writes expected artifacts.455. Check logs for device, observation shape, action shape, and backend errors.4647For skill changes, run:4849```bash50uv run --no-project python tools/skills/cli.py check51```5253## Maintenance5455Keep this skill synchronized with `docs/source/concepts/reinforcement_learning.rst`, `docs/source/isaaclab/installation.rst`, RL training tutorials under `docs/source/how-to/`, and agent configs under `source/isaaclab_tasks/isaaclab_tasks/`. If framework commands or config formats change, update the official training docs or maintained examples first.5657## References5859- [Evaluations](evaluations.md)60- [Examples](examples.md)61- [Debug RL training skill](../debug-rl-training/SKILL.md)62- [Multi-GPU training skill](../train-multi-gpu/SKILL.md)63- [RL training guide](../../../docs/source/concepts/reinforcement_learning.rst)64- [Configure RL training tutorial](../../../docs/source/how-to/configuring_rl_training.rst)65- [Run RL training tutorial](../../../docs/source/how-to/run_rl_training.rst)66- [Task examples](../../../source/isaaclab_tasks/isaaclab_tasks)