Pokemon Player

Operational Steps

1. 确认输入参数完整
2. 执行核心操作（参考本目录下的 scripts/ 或 references/）
3. 验证输出符合契约
4. 保存结果并报告

IO_CONTRACT

• input: game_state: str, actions: list — 用户请求描述、上下文信息
• output: game_response: dict — 宝可梦游戏响应

对应原则：P2（机械原子暴露输入输出规范）

Pokemon Player

Play Pokemon games via headless emulation using the pokemon-agent package.

When to Use

• User says "play pokemon", "start pokemon", "pokemon game"
• User asks about Pokemon Red, Blue, Yellow, FireRed, etc.
• User wants to watch an AI play Pokemon
• User references a ROM file (.gb, .gbc, .gba)

Startup Procedure

1. First-time setup (clone, venv, install)

The repo is NousResearch/pokemon-agent on GitHub. Clone it, then set up a Python 3.10+ virtual environment. Use uv (preferred for speed) to create the venv and install the package in editable mode with the pyboy extra. If uv is not available, fall back to python3 -m venv + pip.

On this machine it is already set up at /home/teknium/pokemon-agent with a venv ready — just cd there and source .venv/bin/activate.

You also need a ROM file. Ask the user for theirs. On this machine one exists at roms/pokemon_red.gb inside that directory. NEVER download or provide ROM files — always ask the user.

2. Start the game server

From inside the pokemon-agent directory with the venv activated, run pokemon-agent serve with --rom pointing to the ROM and --port 9876. Run it in the background with &. To resume from a saved game, add --load-state with the save name. Wait 4 seconds for startup, then verify with GET /health.

3. Set up live dashboard for user to watch

Use an SSH reverse tunnel via localhost.run so the user can view the dashboard in their browser. Connect with ssh, forwarding local port 9876 to remote port 80 on nokey@localhost.run. Redirect output to a log file, wait 10 seconds, then grep the log for the .lhr.life URL. Give the user the URL with /dashboard/ appended. The tunnel URL changes each time — give the user the new one if restarted.

Save and Load

When to save

• Every 15-20 turns of gameplay
• ALWAYS before gym battles, rival encounters, or risky fights
• Before entering a new town or dungeon
• Before any action you are unsure about

How to save

POST /save with a descriptive name. Good examples: before_brock, route1_start, mt_moon_entrance, got_cut

How to load

POST /load with the save name.

List available saves

GET /saves returns all saved states.

Loading on server startup

Use --load-state flag when starting the server to auto-load a save. This is faster than loading via the API after startup.

The Gameplay Loop

Step 1: OBSERVE — check state AND take a screenshot

GET /state for position, HP, battle, dialog. GET /screenshot and save to /tmp/pokemon.png, then use vision_analyze. Always do BOTH — RAM state gives numbers, vision gives spatial awareness.

Step 2: ORIENT

• Dialog/text on screen → advance it
• In battle → fight or run
• Party hurt → head to Pokemon Center
• Near objective → navigate carefully

Step 3: DECIDE

Priority: dialog > battle > heal > story objective > training > explore

Step 4: ACT — move 2-4 steps max, then re-check

POST /action with a SHORT action list (2-4 actions, not 10-15).

Step 5: VERIFY — screenshot after every move sequence

Take a screenshot and use vision_analyze to confirm you moved where intended. This is the MOST IMPORTANT step. Without vision you WILL get lost.

Step 6: RECORD progress to memory with PKM: prefix

Step 7: SAVE periodically

Action Reference

• press_a — confirm, talk, select
• press_b — cancel, close menu
• press_start — open game menu
• walk_up/down/left/right — move one tile
• hold_b_N — hold B for N frames (use for speeding through text)
• wait_60 — wait about 1 second (60 frames)
• a_until_dialog_end — press A repeatedly until dialog clears

Critical Tips from Experience

USE VISION CONSTANTLY

• Take a screenshot every 2-4 movement steps
• The RAM state tells you position and HP but NOT what is around you
• Ledges, fences, signs, building doors, NPCs — only visible via screenshot
• Ask the vision model specific questions: "what is one tile north of me?"
• When stuck, always screenshot before trying random directions

Warp Transitions Need Extra Wait Time

When walking through a door or stairs, the screen fades to black during the map transition. You MUST wait for it to complete. Add 2-3 wait_60 actions after any door/stair warp. Without waiting, the position reads as stale and you will think you are still in the old map.

Building Exit Trap

When you exit a building, you appear directly IN FRONT of the door. If you walk north, you go right back inside. ALWAYS sidestep first by walking left or right 2 tiles, then proceed in your intended direction.

Dialog Handling

Gen 1 text scrolls slowly letter-by-letter. To speed through dialog, hold B for 120 frames then press A. Repeat as needed. Holding B makes text display at max speed. Then press A to advance to the next line. The a_until_dialog_end action checks the RAM dialog flag, but this flag does not catch ALL text states. If dialog seems stuck, use the manual hold_b + press_a pattern instead and verify via screenshot.

Ledges Are One-Way

Ledges (small cliff edges) can only be jumped DOWN (south), never climbed UP (north). If blocked by a ledge going north, you must go left or right to find the gap around it. Use vision to identify which direction the gap is. Ask the vision model explicitly.

Navigation Strategy

• Move 2-4 steps at a time, then screenshot to check position
• When entering a new area, screenshot immediately to orient
• Ask the vision model "which direction to [destination]?"
• If stuck for 3+ attempts, screenshot and re-evaluate completely
• Do not spam 10-15 movements — you will overshoot or get stuck

Running from Wild Battles

On the battle menu, RUN is bottom-right. To reach it from the default cursor position (FIGHT, top-left): press down then right to move cursor to RUN, then press A. Wrap with hold_b to speed through text/animations.

Battling (FIGHT)

On the battle menu FIGHT is top-left (default cursor position). Press A to enter move selection, A again to use the first move. Then hold B to speed through attack animations and text.

Battle Strategy

Decision Tree

1. Want to catch? → Weaken then throw Poke Ball
2. Wild you don't need? → RUN
3. Type advantage? → Use super-effective move
4. No advantage? → Use strongest STAB move
5. Low HP? → Switch or use Potion

Gen 1 Type Chart (key matchups)

• Water beats Fire, Ground, Rock
• Fire beats Grass, Bug, Ice
• Grass beats Water, Ground, Rock
• Electric beats Water, Flying
• Ground beats Fire, Electric, Rock, Poison
• Psychic beats Fighting, Poison (dominant in Gen 1!)

Gen 1 Quirks

• Special stat = both offense AND defense for special moves
• Psychic type is overpowered (Ghost moves bugged)
• Critical hits based on Speed stat
• Wrap/Bind prevent opponent from acting
• Focus Energy bug: REDUCES crit rate instead of raising it

Memory Conventions

Progression Milestones

• Choose starter
• Deliver Parcel from Viridian Mart, receive Pokedex
• Boulder Badge — Brock (Rock) → use Water/Grass
• Cascade Badge — Misty (Water) → use Grass/Electric
• Thunder Badge — Lt. Surge (Electric) → use Ground
• Rainbow Badge — Erika (Grass) → use Fire/Ice/Flying
• Soul Badge — Koga (Poison) → use Ground/Psychic
• Marsh Badge — Sabrina (Psychic) → hardest gym
• Volcano Badge — Blaine (Fire) → use Water/Ground
• Earth Badge — Giovanni (Ground) → use Water/Grass/Ice
• Elite Four → Champion!

Stopping Play

1. Save the game with a descriptive name via POST /save
2. Update memory with PKM:PROGRESS
3. Tell user: "Game saved as [name]! Say 'play pokemon' to resume."
4. Kill the server and tunnel background processes

Pitfalls

-

-

Verification

-

-

• NEVER download or provide ROM files
• Do NOT send more than 4-5 actions without checking vision
• Always sidestep after exiting buildings before going north
• Always add wait_60 x2-3 after door/stair warps
• Dialog detection via RAM is unreliable — verify with screenshots
• Save BEFORE risky encounters
• The tunnel URL changes each time you restart it

验证清单 · VERIFICATION

• 已获取用户 ROM 文件（从未下载或代用户生成 ROM）
• pokemon-agent serve 启动后 GET /health 返回健康（含 --load-state 续档场景）
• 每 2-4 步移动后截图并用 vision_analyze 核实位置，未连续发送 >4-5 个动作
• 门/楼梯换图后追加 2-3 个 wait_60，出建筑后先横向 2 格再行进
• 道馆战/新区域等高风险操作前已用描述性名称 POST /save
• 停止时已保存游戏、更新 PKM:PROGRESS 并终止 serve 与 localhost.run 隧道进程

约束规则 · RULES

1. 输入约束: 参数类型、范围、格式必须校验
2. 输出约束: 返回值结构、编码、命名必须一致
3. 异常约束: 错误信息必须包含上下文和恢复建议
4. 安全约束: 不执行未验证的任意代码，不暴露内部状态

Golden 集合 · GOLDEN SET

• Golden Input: 标准输入样本（覆盖正常路径）
• Golden Output: 预期输出（精确匹配或格式校验）
• Golden Error: 预期错误信息（覆盖失败路径）

Golden 集合是测试的单一真理来源。所有改进必须通过 golden 测试。

违反规则的操作视为不安全，必须拒绝或隔离。

每项验证必须可执行、可记录、可复现。验证失败时记录原因和修复。

Genes (策略基因)

紧凑策略表示。条件→策略。需要深度时参考完整文档。

• [POKE-001] 执行移动或导航操作 → 每 2-4 步必须截图并使用视觉分析验证位置，严禁仅依赖 RAM 状态数据
• [POKE-002] 遭遇对话框或文本滚动 → 使用 hold_b 加速文本显示后按 a 推进，若卡住则手动组合按键并截图验证
• [POKE-003] 通过门或楼梯进行地图切换 → 在动作序列末尾添加 2-3 个 wait_60 以等待过渡完成，防止读取到过时的位置信息
• [POKE-004] 从建筑物出口走出 → 先向左或右横向移动 2 格避开门口，再执行预定方向的移动，防止立即走回室内
• [POKE-005] 遇到向北被悬崖（Ledge）阻挡 → 利用视觉模型识别缺口方向，仅允许向南跳跃，需横向寻找绕行路径
• [POKE-006] 面临战斗决策且无类型优势 → 优先使用最强 STAB（本系）招式，若 HP 过低则切换宝可梦或使用药水
• [POKE-007] 执行高风险操作（道馆战、新区域、不确定动作）前 → 必须使用描述性名称（如 before_brock）保存游戏状态
• [POKE-008] 连续 3 次尝试移动失败或陷入僵局 → 立即停止随机尝试，截图重新评估环境并更新记忆中的 PKM:STUCK 记录