PufferLib v2
Use this skill for codebases pinned to PufferLib 2.x (pufferlib>=2.0,<3.0). PufferLib 2.0 was released on PyPI on December 8, 2024 and introduced the Ocean/native-environment generation described by PufferAI as the 1M SPS release.
[!IMPORTANT]
This is not the v1 API. Do not write from pufferlib import PuffeRL; import the training module with from pufferlib import pufferl or use the puffer CLI. If the repository already uses trainer.evaluate(), trainer.train(), or trainer.mean_and_log(), inspect imports and pinning before editing because that pattern may be v1 or early-v2 code.
Version Selection
- Prefer
pufferlib-v2 only for projects pinned to pufferlib>=2.0,<3.0, reproducing 2024/early-2025 experiments, or maintaining PufferLib 2.x environments.
- Prefer
pufferlib-v3 for pufferlib>=3.0,<4.0 projects or when the code uses newer pufferlib.pufferl config helpers.
- Prefer the legacy
pufferlib skill only for pufferlib==1.0.0 code.
Installation
python -m venv .venv
source .venv/bin/activate
pip install "pufferlib>=2.0,<3.0"
For environments with native graphics, Atari, NetHack, Procgen, Neural MMO, or CUDA-sensitive dependencies, prefer the upstream PufferTank container or the repository's pinned environment. Keep Python, PyTorch, CUDA, Gymnasium, and PettingZoo versions synchronized with the project lockfile.
Primary Workflow
- Confirm the version:
python - <<'PY'
import pufferlib, importlib.metadata
print(importlib.metadata.version('pufferlib'))
print(pufferlib.__file__)
PY
- Prefer CLI training for supported Ocean/built-in environments:
puffer train ENV_NAME --help
puffer train ENV_NAME
puffer eval ENV_NAME --help
- For Python training, prefer the module-level
pufferl API and local examples in the target repository over the old top-level PuffeRL import.
- Keep vectorized environment settings explicit: number of workers, environments per worker / environment count, batch size, backend, and async/sync mode.
- Validate with a tiny run before launching a long experiment.
Native Environment and Vectorization Guidance
PufferLib 2.x centers on high-throughput simulation:
- Use native PufferEnv/Ocean environments when performance matters. Native envs can write observations directly into buffers used by vectorization/training.
- Keep Gymnasium/PettingZoo support as a compatibility layer. Wrap existing environments rather than rewriting them first.
- Use a VecEnv-style interface by default. Async operation extends the synchronous step/reset workflow, so implement synchronous logic first, then add send/recv-style async only if needed.
- Start with a conservative worker count. Increase workers/envs per worker only after profiling CPU utilization and policy-forward time.
Training Pattern
Use this shape for v2 maintenance work; adapt names from the installed package and repository examples rather than assuming v1 signatures:
from pufferlib import pufferl
# Prefer repository-provided config/env/policy helpers when available.
config = load_or_create_config()
vecenv = make_vectorized_env(config)
policy = make_policy(vecenv, config)
trainer = pufferl.PuffeRL(config, vecenv, policy)
while not done_training(trainer, config):
trainer.evaluate()
logs = trainer.train()
if logs:
handle_logs(logs)
checkpoint = trainer.close()
When migrating old v1 code:
- Replace
from pufferlib import PuffeRL with from pufferlib import pufferl.
- Replace
pufferlib.make(...) calls with the v2 vectorization/environment construction used by the project.
- Ensure policies accept the state argument expected by the trainer when recurrent state or async environment metadata is enabled.
- Treat
mean_and_log() as trainer-internal unless local examples call it directly.
Environment Development Checklist
- Define observation/action spaces before reset/step data are consumed.
- Return consistent Gymnasium-style
obs, reward, terminated, truncated, info data in wrappers, or the native PufferEnv equivalent in native envs.
- For multi-agent envs, keep agent ordering stable and document how dead/missing agents are padded or masked.
- Avoid per-step allocation in performance-critical environments. Allocate buffers at initialization and write in place.
- Add a smoke test that resets, steps random actions for at least 100 transitions, and checks shapes/dtypes.
Performance Checklist
- Profile environment step time separately from policy forward/backward time.
- Use async vectorization when policy inference and environment stepping can overlap.
- Keep observation copies to a minimum; prefer native buffers for Ocean/C environments.
- Tune
num_workers, env count, rollout horizon, batch size, and minibatch size together.
- Compare SPS after warmup, not during first-epoch compilation/build overhead.
Common Failure Modes
- ImportError for
PuffeRL: old v1 import; use from pufferlib import pufferl.
- Shape mismatch on first batch: wrapper spaces do not match reset/step output; print
single_observation_space, single_action_space, and first observation dtype/shape.
- Hanging multiprocessing run: test the same env with a serial backend, then reduce workers to one and re-enable multiprocessing.
- Low SPS with fast native env: check policy forward time, Python callbacks/logging, info payload size, and whether async mode is enabled.
Sources to Check When Updating
- PyPI release history for exact 2.x versions.
- PufferAI blog post “PufferLib 2.0: Reinforcement Learning at 1,000,000 Steps/Second”.
- The target repository's lockfile and training examples; v2 projects often carried local helper wrappers.
1---2name: pufferlib-v23description: PufferLib 2.x reinforcement learning workflows for the Dec 2024 API generation. Use when working with pufferlib>=2.0,<3.0, Puffer Ocean C environments, native PufferEnv/VecEnv-style vectorization, Gymnasium/PettingZoo compatibility wrappers, asynchronous sampling, or v2 training/evaluation migration from the deprecated v1 PuffeRL top-level API.4metadata:5 source: https://skillmd.com/skills/stanfish06/pufferlib-v26---78# PufferLib v2910Use this skill for codebases pinned to **PufferLib 2.x** (`pufferlib>=2.0,<3.0`). PufferLib 2.0 was released on PyPI on **December 8, 2024** and introduced the Ocean/native-environment generation described by PufferAI as the 1M SPS release.1112> [!IMPORTANT]13> This is not the v1 API. Do **not** write `from pufferlib import PuffeRL`; import the training module with `from pufferlib import pufferl` or use the `puffer` CLI. If the repository already uses `trainer.evaluate()`, `trainer.train()`, or `trainer.mean_and_log()`, inspect imports and pinning before editing because that pattern may be v1 or early-v2 code.1415## Version Selection1617- Prefer `pufferlib-v2` only for projects pinned to `pufferlib>=2.0,<3.0`, reproducing 2024/early-2025 experiments, or maintaining PufferLib 2.x environments.18- Prefer `pufferlib-v3` for `pufferlib>=3.0,<4.0` projects or when the code uses newer `pufferlib.pufferl` config helpers.19- Prefer the legacy `pufferlib` skill only for `pufferlib==1.0.0` code.2021## Installation2223```bash24python -m venv .venv25source .venv/bin/activate26pip install "pufferlib>=2.0,<3.0"27```2829For environments with native graphics, Atari, NetHack, Procgen, Neural MMO, or CUDA-sensitive dependencies, prefer the upstream PufferTank container or the repository's pinned environment. Keep Python, PyTorch, CUDA, Gymnasium, and PettingZoo versions synchronized with the project lockfile.3031## Primary Workflow32331. Confirm the version:34 ```bash35 python - <<'PY'36 import pufferlib, importlib.metadata37 print(importlib.metadata.version('pufferlib'))38 print(pufferlib.__file__)39 PY40 ```412. Prefer CLI training for supported Ocean/built-in environments:42 ```bash43 puffer train ENV_NAME --help44 puffer train ENV_NAME45 puffer eval ENV_NAME --help46 ```473. For Python training, prefer the module-level `pufferl` API and local examples in the target repository over the old top-level `PuffeRL` import.484. Keep vectorized environment settings explicit: number of workers, environments per worker / environment count, batch size, backend, and async/sync mode.495. Validate with a tiny run before launching a long experiment.5051## Native Environment and Vectorization Guidance5253PufferLib 2.x centers on high-throughput simulation:5455- Use native PufferEnv/Ocean environments when performance matters. Native envs can write observations directly into buffers used by vectorization/training.56- Keep Gymnasium/PettingZoo support as a compatibility layer. Wrap existing environments rather than rewriting them first.57- Use a VecEnv-style interface by default. Async operation extends the synchronous step/reset workflow, so implement synchronous logic first, then add send/recv-style async only if needed.58- Start with a conservative worker count. Increase workers/envs per worker only after profiling CPU utilization and policy-forward time.5960## Training Pattern6162Use this shape for v2 maintenance work; adapt names from the installed package and repository examples rather than assuming v1 signatures:6364```python65from pufferlib import pufferl6667# Prefer repository-provided config/env/policy helpers when available.68config = load_or_create_config()69vecenv = make_vectorized_env(config)70policy = make_policy(vecenv, config)7172trainer = pufferl.PuffeRL(config, vecenv, policy)73while not done_training(trainer, config):74 trainer.evaluate()75 logs = trainer.train()76 if logs:77 handle_logs(logs)7879checkpoint = trainer.close()80```8182When migrating old v1 code:8384- Replace `from pufferlib import PuffeRL` with `from pufferlib import pufferl`.85- Replace `pufferlib.make(...)` calls with the v2 vectorization/environment construction used by the project.86- Ensure policies accept the state argument expected by the trainer when recurrent state or async environment metadata is enabled.87- Treat `mean_and_log()` as trainer-internal unless local examples call it directly.8889## Environment Development Checklist9091- Define observation/action spaces before reset/step data are consumed.92- Return consistent Gymnasium-style `obs, reward, terminated, truncated, info` data in wrappers, or the native PufferEnv equivalent in native envs.93- For multi-agent envs, keep agent ordering stable and document how dead/missing agents are padded or masked.94- Avoid per-step allocation in performance-critical environments. Allocate buffers at initialization and write in place.95- Add a smoke test that resets, steps random actions for at least 100 transitions, and checks shapes/dtypes.9697## Performance Checklist9899- Profile environment step time separately from policy forward/backward time.100- Use async vectorization when policy inference and environment stepping can overlap.101- Keep observation copies to a minimum; prefer native buffers for Ocean/C environments.102- Tune `num_workers`, env count, rollout horizon, batch size, and minibatch size together.103- Compare SPS after warmup, not during first-epoch compilation/build overhead.104105## Common Failure Modes106107- **ImportError for `PuffeRL`**: old v1 import; use `from pufferlib import pufferl`.108- **Shape mismatch on first batch**: wrapper spaces do not match reset/step output; print `single_observation_space`, `single_action_space`, and first observation dtype/shape.109- **Hanging multiprocessing run**: test the same env with a serial backend, then reduce workers to one and re-enable multiprocessing.110- **Low SPS with fast native env**: check policy forward time, Python callbacks/logging, info payload size, and whether async mode is enabled.111112## Sources to Check When Updating113114- PyPI release history for exact 2.x versions.115- PufferAI blog post “PufferLib 2.0: Reinforcement Learning at 1,000,000 Steps/Second”.116- The target repository's lockfile and training examples; v2 projects often carried local helper wrappers.