# Review Docs

> Audit a game environment's README.md and AGENTS.md against its engine implementation. Use when the user asks to "review", "audit", "check", or "fact-check" environment docs, when players report doc/engine discrepancies, or before a competition launch. Also use proactively after changing an environment's engine (`kaggle_environments/envs/<game>/<game>.py`) when a sibling README.md or AGENTS.md exists — rebalances and mechanic changes routinely leave those docs stale. The engine is always the source of truth — docs get fixed, not the engine.

- Skill: `kaggle/review-docs` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kaggle/review-docs`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kaggle/review-docs/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Security
- Author: kaggle (https://skillmd.com/u/kaggle)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/kaggle/review-docs

---


# Review Environment Docs Against the Engine

Audit `kaggle_environments/envs/<game>/README.md` and `AGENTS.md` for claims that disagree with the engine.

## Core rule

**The engine is the source of truth. Fix the docs, not the engine.** Agents play against the implementation, and changing engine behavior late invalidates existing submissions and replays.

Exception: if a mechanic is clearly a *bug* rather than an undocumented quirk, flag it and ask. Don't silently enshrine a bug as intended.

## Method: simulate, don't reason

Never verify a claim by reading code and reasoning about it. Execute the engine and observe. Cheapest entry point first:

```python
# 1. Internal helpers — isolates one rule
from kaggle_environments.envs.<game> import <game> as G
G._new_<entity>(...)      # constructor: initial field values
G._apply_<action>(...)    # one action
G._<periodic_hook>(...)   # end-of-turn / round / phase transition

# 2. Full env — catches validation the helpers skip
from kaggle_environments import make
env = make("<game>", configuration={...}); env.reset(2); env.step([a0, a1])

# 3. OpenSpiel games
import pyspiel; state = pyspiel.load_game("<name>").new_initial_state()
```

Step through the **entire lifecycle** — creation to terminal state — recording observable state at every step, not just the endpoints. Most doc bugs live in the middle: a value that plateaus early, a counter starting at the wrong number, a transition firing a step sooner than the prose implies.

Gotchas:
- Don't name scratch files after stdlib modules (`numbers.py` breaks the import chain).
- Helpers may *replace* an object rather than mutate it (one entity type becoming another). Re-read from the container after each call.
- Verify action- and transaction-related claims through `make()` too.

## Checklist

Skip sections that don't apply. Each item is a real bug class found in a prior audit.

### Constants, tables, and formulas
- [ ] Every value in a doc table matches the engine's constants. Check every row, not a sample.
- [ ] Derived columns (rates, per-unit costs) use one consistent formula, and the doc says which. Mixed formulas across rows is the tell.
- [ ] "Max" values are reachable. A cap needing an optional booster needs that qualifier; a cap unreachable in a variant lacking the booster is wrong.
- [ ] Column headers mean what the values show. If they disagree, ask the user whether to change the number or the header.
- [ ] Formulas match the code symbol for symbol — exponents, log bases, clamps, order of operations. Recompute any worked example.

### State initialization and lifecycle
- [ ] Initial value of every field in each constructor. A counter starting at 1 instead of 0 silently removes a grace period the prose implies.
- [ ] "Ongoing" / "indefinite" / "repeating" claims: find the cap (a count-vs-max guard, a lifespan field being armed). If one exists, the doc must say so.
- [ ] Exact step an entity or episode ends, and what stays usable on the way out.
- [ ] Ordering inside periodic hooks: which mutations run before which checks. That ordering is often what makes a rule surprising.
- [ ] Cooldowns: when the counter ticks relative to the action, whether a failed attempt consumes it, turns-between vs. turns-until.
- [ ] Rates that ramp or decay: recompute at start, middle, and end. Watch for rounding or a clamp that makes the stated endpoint arrive early.
- [ ] Entities that change type in place (transforming, upgrading, being captured) — what carries over, what resets.

### Actions and preconditions
- [ ] Walk each documented action against its handler. Every early `return`/`continue` is an undocumented precondition.
- [ ] Partial success: an action that works in one state but no-ops in a neighbouring one needs the qualifier.
- [ ] No-op semantics: does a rejected action still consume the resource, turn, or cooldown?
- [ ] Legality rules for movement, placement, targeting — and whether spawn/setup logic obeys the same rules the player does.
- [ ] Malformed input: wrong arity, wrong type, out-of-range. Silently dropping one move from a submitted list is a rule players need.
- [ ] Per-turn action limits and what happens to the excess.

### Turn pipeline and interaction resolution
- [ ] If the doc lists a numbered turn order, walk the engine top to bottom and confirm each stage's position. Late-inserted stages (cleanup, spawning, expiry) get omitted.
- [ ] Where an entity is mutated relative to where it's read. Spawned after the action phase → unusable until next turn; removed before it → never usable.
- [ ] Contested outcomes (combat, collision, simultaneous claims): work ties, three-way contests, and zero-survivor cases by hand. Docs describe only the two-party win case.
- [ ] Continuous vs. endpoint-sampled detection (swept paths vs. position checks). Changes which interactions are possible; state it outright.
- [ ] Precedence when several removal conditions fire on one entity in the same step.

### Procedural generation and randomness
- [ ] Documented ranges match the generator's bounds, including any retry cap that can silently under-deliver.
- [ ] Guarantees ("at least one of X", "always symmetric") are enforced by code, not merely likely. Find the loop that makes them true, or drop "guaranteed".
- [ ] Distribution claims ("skewed low", "weighted toward") imply specific sampling — verify rather than restate.
- [ ] What the seed controls, and whether it's scrubbed from the observation before agents see it.

### Hidden state and partial observability
- [ ] Fields agents are told to use for prediction contain what's claimed, and stay valid as the episode progresses.
- [ ] Anything the doc says is derivable, derive it: recompute from the observation alone and compare to engine state.
- [ ] Information agents should *not* have is genuinely absent — check what the **agent receives**, not the shared state object. Engines routinely keep global state on `state[0].observation` and strip it per player. Run an agent that dumps its own observation keys.
- [ ] If visibility is asymmetric, confirm the players' observations actually differ and each sees only its share.
- [ ] Per-category persistence. In a "visible now / remembered later" table every row is a separate claim, and "remembered" may mean last-known-value rather than live.
- [ ] Sentinels for unknown data (`-1`, `None`, absent key) are documented and mean *unknown*, not a legitimate zero.

### Termination and scoring
- [ ] Every termination condition, not just the step limit. Elimination, stalemate, and early-exit branches get missed.
- [ ] The step the episode actually ends on. Off-by-one against `episodeSteps` is common; run to completion.
- [ ] The reward the interpreter assigns, including ties and all-players-eliminated. Does the doc's "winner" language match a win/loss constant or a raw score?
- [ ] Rewards that change meaning mid-episode (running score during play, win/loss constant at the end) need both forms documented.
- [ ] Multi-stage tiebreakers: each stage exists, in that order, with the stated final fallback.
- [ ] What counts toward the final score and what's excluded — value in intermediate state often doesn't count.

### Observation and action format
- [ ] Every documented field exists, with that type and meaning.
- [ ] Field comments describing *when* something is set: verify the trigger. "Set after X" is often wrong when the engine sets it unconditionally.
- [ ] Boolean vs. counter. A boolean means it doesn't accumulate — say so if a player might expect stockpiling.
- [ ] Anything the docs imply lives in the state structure but doesn't. Agents will search and find nothing; document the coordinates or accessor instead.
- [ ] Documented action strings and argument shapes match what the handler parses.

### Configuration
- [ ] Doc defaults match the `.json` specification.
- [ ] **Every documented knob is actually read.** Diff the doc table against the `.json` keys, then grep each in the engine. A knob ignored in favour of a module constant is worse than undocumented — players tune it and see no effect.
- [ ] Claims holding only at default config are labelled as such.
- [ ] Derived figures quoted in prose recompute from the stated defaults.

### Resource and economy systems (if present)
- [ ] Which items/actions are permitted on each side of a transaction. Check membership tests (`x in SOME_LIST`) separately from explicit whitelists — an unfiltered membership test permits more than the author may have intended.
- [ ] Calibration constants: confirm the stated derivation reproduces them. If not, they're stale from a rebalance or the explanation is wrong.
- [ ] Any horizon or reference quantity differing from the episode length needs a reason in the doc.

### Cross-file consistency
- [ ] README and AGENTS.md agree. When they conflict, both may still be wrong — check the engine.
- [ ] Beginner/advanced or base/arena variants have their own engines. A fix in one usually applies to the sibling, but *verify against that sibling's engine* — constants often differ.
- [ ] Module-level comments and docstrings in the engine are docs too, and drift.

## Style

Docs are declarative rulebooks, not strategy guides.

- State the rule; don't advise. Say what the mechanic does, not what the player should do about it.
- No "you should", "plan on", "budget for", "make sure", "worth noting".
- No bold lead-ins to make a rule feel urgent.
- Leave pre-existing prose alone unless it's factually wrong. Rewriting a doc's voice is out of scope; if a paragraph is hard to read, a paragraph break beats an edit.

## Reporting

Group findings as **confirmed** / **overstated** / **rejected**. For each: file:line, what the engine does, how you verified it. Separate doc bugs from engine bugs — engine bugs need the user's decision, not your fix.

Surface explicitly:
- **Bugs the reporter missed.** A user-filed list is a starting point, not a scope.
- **Judgment calls**, where the engine's behavior looks unintentional. Documenting it is the default, but say so plainly and give the alternative fix.

Before finishing: re-verify every number you wrote with one audit script asserting doc value == engine value, and run the env's test suite.