Training Pipeline

Purpose

Entry point for RL training tasks. Routes to specialized skills.

0. Check AutoML and Run Directories

# 1. List all AutoML runs and their outcomes
Get-ChildItem starter_kit_log/automl_* -Directory | ForEach-Object {
    Write-Host "`n=== $($_.Name) ===" -ForegroundColor Cyan
    $state = Join-Path $_.FullName "state.yaml"
    if (Test-Path $state) { Get-Content $state | Select-Object -First 30 }
}

# 2. Check current automl progress state
if (Test-Path starter_kit_schedule/progress/automl_state.yaml) {
    Get-Content starter_kit_schedule/progress/automl_state.yaml
}

# 3. Read task-specific reference (reward scales, search spaces, terrain, etc.)
Get-Content starter_kit_docs/<task-name>/Task_Reference.md

# 4. List training runs and their timestamps
Get-ChildItem runs/<env-name>/ -Directory | Sort-Object Name -Descending | Select-Object -First 10

# 4. Review reward library for tried components
Get-ChildItem starter_kit_schedule/reward_library/ -Recurse -Filter "*.yaml" -ErrorAction SilentlyContinue

# 5. Check WAKE_UP.md if progress_watcher was running  
if (Test-Path starter_kit_schedule/progress/WAKE_UP.md) {
    Get-Content starter_kit_schedule/progress/WAKE_UP.md
}

What to look for:

• Best reward/composite score achieved so far
• Which HP configurations worked best
• Known failure modes already diagnosed
• Whether anti-laziness mechanisms are active in the current code

Only after reviewing, proceed to Quick Start commands below.

Quick Start — Just Run Training

🔴 AutoML-First Policy (MANDATORY): See .github/copilot-instructions.md for the full policy. NEVER use train.py for parameter exploration — use automl.py batch search. train.py is ONLY for: smoke tests (<500K steps), --render visual debug, or final deployment runs with known-good config.

# === PRIMARY: AUTOML PIPELINE (USE THIS for all parameter exploration) ===
uv run starter_kit_schedule/scripts/automl.py `
    --mode stage `
    --budget-hours 8 `
    --hp-trials 15

# === MONITOR AUTOML STATE ===
Get-Content starter_kit_schedule/progress/automl_state.yaml

# === READ AUTOML RESULTS ===
Get-Content starter_kit_log/automl_*/report.md

# === SMOKE TEST ONLY (<500K steps, verify code compiles) ===
uv run scripts/train.py --env <env-name> --train-backend torch --max-env-steps 200000 --check-point-interval 50

# === VISUAL DEBUGGING
uv run scripts/train.py --env <env-name> --train-backend torch --render

# === FINAL DEPLOYMENT RUN (after AutoML found best config) ===
uv run scripts/train.py --env <env-name> --train-backend torch

# === EVALUATE A CHECKPOINT ===
uv run scripts/play.py --env <env-name>

# === TENSORBOARD ===
uv run tensorboard --logdir runs/<env-name>

Hardware Performance (measured)

Task-Specific Reference

Environment-specific details (reward scales, search spaces, terrain descriptions, known exploits, empirical findings) are in: starter_kit_docs/{task-name}/Task_Reference.md

Experiment history is in: starter_kit_docs/{task-name}/REPORT_*.md

These contain critical knowledge about reward hacking exploits (e.g., lazy robot), anti-laziness mechanisms, dead gradient fixes, sprint-crash exploits, and LR scheduler findings.

Skill Routing

Key Pipeline Scripts

When to Read Each Skill

Known Issues (Already Fixed)

These are pipeline-level issues that have been resolved. Do NOT re-investigate:

Task-specific known issues (reward exploits, dead gradients, LR scheduler findings) are documented in starter_kit_docs/{task}/Task_Reference.md.

Managed Directories

---
> Converted and distributed by [TomeVault](https://tomevault.io/claim/mzqef) — claim your Tome and manage your conversions.
<!-- tomevault:4.0:skill_md:2026-04-16 -->