Machine Learning Systems Engineer, RL Engineering
When to Use
- Design RL training platform — controllers, workers, resource scheduling
- Implement rollout collection — vectorized envs, async actors, trajectory buffers
- Operate distributed training — data parallel, parameter servers, gradient sync patterns
- Manage replay buffers — prioritization, storage, sampling at scale
- Wire checkpointing — policy/value nets, optimizer state, resume after preemption
- Integrate experiment tracking — seeds, configs, metric schemas, artifact lineage
- Connect simulators — Gymnasium-style APIs, custom env servers, batch stepping
- Export policies for batch eval or downstream inference path
- Debug training instability — NaNs, reward scale, worker desync, straggler GPUs
- Plan GPU/memory layout for actor vs learner processes
When NOT to Use
- Churn models, A/B tests, classical supervised pipelines →
data-scientist
- Production LLM features, agents, RAG →
ai-engineer
- Safeguard/moderation inference gateways →
ml-infrastructure-engineer-safeguards
- Safety classifier research →
ml-research-engineer-safeguards
- CI/CD and generic K8s ops →
devops, cluster-deployment-engineer
- DC-wide GPU supply programs →
data-center-compute-supply-efficiency
- HTTP API p99 without RL training context →
performance-engineer
- RL algorithm theory only (no systems) →
ai-researcher for literature; stay systems-focused here
Related skills
| Need |
Skill |
| Supervised ML and statistical eval |
data-scientist |
| General AI research methodology |
ai-researcher |
| Inference gateways and model serving |
ml-infrastructure-engineer-safeguards |
| Training cluster / K8s jobs |
cluster-deployment-engineer |
| Pipelines and GitOps |
devops |
| GPU capacity at facility level |
data-center-compute-supply-efficiency |
| Serving latency and load tests |
performance-engineer |
| Product agents using RL outcomes |
ai-engineer |
Core Workflows
1. RL systems framing
Env contract, on/off-policy, scale targets.
See references/rl_systems_framing.md.
2. Training platform architecture
Controllers, workers, scheduling.
See references/training_platform_architecture.md.
3. Environments and rollouts
Vectorization, trajectory format.
See references/environments_rollouts.md.
4. Replay, checkpoints, experiments
Buffers, resume, tracking.
See references/replay_checkpoints_experiments.md.
5. Evaluation and policy export
Eval harness, deployment handoff.
See references/evaluation_policy_export.md.
6. Reliability and observability
Stability, metrics, incident debug.
See references/reliability_observability_rl.md.
Outputs
- Architecture doc — actor/learner topology, data flow, failure domains
- Env API spec — observation, action, reward, reset, seed semantics
- Runbook — launch, resume, preempted job recovery, scale-out
- Config template — hyperparameters + infra knobs versioned together
- Metric dashboard spec — reward, length, KL, GPU, steps/sec, queue depth
- Policy export package — weights, normalization stats, eval report
Principles
- Reproducibility — seed envs, log config hash, pin sim versions
- Separate rollout from learn — scale collectors and learners independently
- Deterministic resume — checkpoint includes optimizer and buffer cursor when needed
- Observe the MDP — log reward components, not only scalar return
- Fail fast on desync — version mismatch between workers is a top incident class
1---2name: ml-systems-engineer-rl-engineering3description: Guides ML systems engineering for reinforcement learning—distributed training platforms, rollout workers and vectorized environments, replay buffers, policy/critic serving for train loops, checkpointing and experiment tracking, sim-to-real hooks, and RL training reliability. Use when building RL training infrastructure, scaling PPO/SAC-style jobs, debugging unstable distributed rollouts, designing env APIs, or exporting policies for inference—not for supervised ML product modeling (data-scientist), LLM RAG/agents (ai-engineer), safeguard classifiers (ml-research-engineer-safeguards, ml-infrastructure-engineer-safeguards), general GPU serving without RL (ml-infrastructure-engineer-safeguards), or app latency profiling (performance-engineer).4---56# Machine Learning Systems Engineer, RL Engineering78## When to Use910- Design **RL training platform** — controllers, workers, resource scheduling11- Implement **rollout collection** — vectorized envs, async actors, trajectory buffers12- Operate **distributed training** — data parallel, parameter servers, gradient sync patterns13- Manage **replay buffers** — prioritization, storage, sampling at scale14- Wire **checkpointing** — policy/value nets, optimizer state, resume after preemption15- Integrate **experiment tracking** — seeds, configs, metric schemas, artifact lineage16- Connect **simulators** — Gymnasium-style APIs, custom env servers, batch stepping17- **Export policies** for batch eval or downstream inference path18- Debug **training instability** — NaNs, reward scale, worker desync, straggler GPUs19- Plan **GPU/memory** layout for actor vs learner processes2021## When NOT to Use2223- Churn models, A/B tests, classical supervised pipelines → `data-scientist`24- Production LLM features, agents, RAG → `ai-engineer`25- Safeguard/moderation inference gateways → `ml-infrastructure-engineer-safeguards`26- Safety classifier research → `ml-research-engineer-safeguards`27- CI/CD and generic K8s ops → `devops`, `cluster-deployment-engineer`28- DC-wide GPU supply programs → `data-center-compute-supply-efficiency`29- HTTP API p99 without RL training context → `performance-engineer`30- RL algorithm theory only (no systems) → `ai-researcher` for literature; stay systems-focused here3132## Related skills3334| Need | Skill |35|---|---|36| Supervised ML and statistical eval | `data-scientist` |37| General AI research methodology | `ai-researcher` |38| Inference gateways and model serving | `ml-infrastructure-engineer-safeguards` |39| Training cluster / K8s jobs | `cluster-deployment-engineer` |40| Pipelines and GitOps | `devops` |41| GPU capacity at facility level | `data-center-compute-supply-efficiency` |42| Serving latency and load tests | `performance-engineer` |43| Product agents using RL outcomes | `ai-engineer` |4445## Core Workflows4647### 1. RL systems framing4849Env contract, on/off-policy, scale targets.5051**See `references/rl_systems_framing.md`.**5253### 2. Training platform architecture5455Controllers, workers, scheduling.5657**See `references/training_platform_architecture.md`.**5859### 3. Environments and rollouts6061Vectorization, trajectory format.6263**See `references/environments_rollouts.md`.**6465### 4. Replay, checkpoints, experiments6667Buffers, resume, tracking.6869**See `references/replay_checkpoints_experiments.md`.**7071### 5. Evaluation and policy export7273Eval harness, deployment handoff.7475**See `references/evaluation_policy_export.md`.**7677### 6. Reliability and observability7879Stability, metrics, incident debug.8081**See `references/reliability_observability_rl.md`.**8283## Outputs8485- **Architecture doc** — actor/learner topology, data flow, failure domains86- **Env API spec** — observation, action, reward, reset, seed semantics87- **Runbook** — launch, resume, preempted job recovery, scale-out88- **Config template** — hyperparameters + infra knobs versioned together89- **Metric dashboard spec** — reward, length, KL, GPU, steps/sec, queue depth90- **Policy export package** — weights, normalization stats, eval report9192## Principles9394- **Reproducibility** — seed envs, log config hash, pin sim versions95- **Separate rollout from learn** — scale collectors and learners independently96- **Deterministic resume** — checkpoint includes optimizer and buffer cursor when needed97- **Observe the MDP** — log reward components, not only scalar return98- **Fail fast on desync** — version mismatch between workers is a top incident class
daemon-blockint-tech