Rl Policy Optimization

Best practices for reinforcement learning policy optimization. Use when working on RL agents, PPO, SAC, or reward design.

enuno Updated 356 repo stars

File contents

RL Policy Optimization Best Practice

Algorithm selection:

  • Discrete actions: PPO, DQN, A2C
  • Continuous actions: SAC, TD3, PPO
  • Multi-agent: MAPPO, QMIX
  • Offline: CQL, IQL, Decision Transformer

Training recipe:

  • PPO: clip=0.2, lr=3e-4, gamma=0.99, GAE lambda=0.95
  • SAC: lr=3e-4, tau=0.005, auto-tune alpha
  • Use vectorized environments (e.g., gymnasium.vector)
  • Normalize observations and rewards
  • Log episode return, episode length, value loss, policy entropy

Evaluation:

  • Report mean +/- std over 10+ evaluation episodes
  • Use deterministic policy for evaluation
  • Compare against random policy and simple baselines
  • Report sample efficiency (return vs. env steps)

Common pitfalls:

  • Reward shaping can introduce bias
  • Seed sensitivity is HIGH — use 5+ seeds
  • Hyperparameter sensitivity — do a small sweep

enuno/crewless-capital commit 2ee972f9e6

Frequently asked questions

npx get-skill add enuno/rl-policy-optimization