Verifying Turn Based Games

Turn-Based Game Verification

A reusable contract + measurement methodology for two-player turn-based games.

Two parts:

1. Engine Interface Contract — pure-function shape any conforming engine must satisfy, enabling search, replay, and quality analysis to be written generically.
2. Quality Measurement Methodology — vocabulary and metrics for judging game quality beyond raw win-rates.

This skill prescribes the what (contract and metric definitions). It does not prescribe language / runtime / file layout / build system / tooling commands, ideation or tag workflows, player-facing documentation format, or engine code generation — those are project-local choices.

Scope

In scope:

• Two-player, strict alternating turns
• Perfect or imperfect information
• Finite turn cap
• Deterministic RNG via explicit seed (no side-effect randomness)

Out of scope (do not apply):

• Simultaneous actions, reaction/interrupt windows
• 3+ players
• Real-time / non-turn-based
• Single-player puzzles

If the target game does not fit the in-scope list, stop and tell the user this skill does not apply rather than forcing the contract.

Part 1: Engine Interface Contract

The engine MUST expose these pure functions. Signatures shown in TypeScript-ish notation; translate to the host language as needed.

init(config?: Config, seed?: number): State
step(state: State, action: Action): { state: State, reward?: number, done: boolean, info: object }
getLegalActions(state: State): Action[]
getObservation(state: State, playerId: PlayerId): State
getResult(state: State): { done: boolean, winners: PlayerId[], reason: string, scores?: Record<PlayerId, number> }
scoreState(state: State, playerId: PlayerId): number
replayLog(state: State): string
cloneState?(state: State): State

Contract notes:

• getLegalActions and step use state.currentPlayer implicitly. The contract assumes strict alternating turns; do not extend it to simultaneous play.
• state.history: Event[] MUST exist and MUST be appended to inside step. replayLog is a pure formatter over state.history; no external collector.
• state.rngSeed MUST exist; all randomness inside step derives from it deterministically. No Math.random(), clock reads, or other side-effect randomness.
• getObservation returns the same State type with hidden fields nulled or masked, NOT a different observation type. This keeps bots type-compatible with the engine.
• getResult replaces a simpler getWinner so callers can distinguish win-by-HP / win-by-deck-out / draw-by-turn-cap. The reason string is required.
• scoreState is required by the measurement layer (Part 2). An engine that omits scoreState cannot be quality-evaluated by this skill.
• cloneState is only required when State is not fully immutable. If the host language guarantees immutability (or you use structural sharing), omit it.

Required invariants

• No global mutable state.
• step does not mutate its input state. Branch search via step(cloneState(s), a) or rely on immutability.
• Replay determinism: given a fixed seed and a fixed action sequence, init followed by repeated step produces byte-identical states across runs.
• Single illegal-action policy: choose ONE and document it in the engine — throw, return { state: s, done: false, info: { illegal: true } } without state change, or auto-pass. Do not mix policies.
• Action and State must be JSON-serializable (replay, persistence, search caching all depend on this).

Pre-implementation checklist

Before writing the engine, the rules document must specify:

• Exact data model for State (turn, currentPlayer, public board, private state, rngSeed, history)
• All terminal conditions in priority order, each with its reason string
• Action schema and preconditions for each action type
• Pass behavior when getLegalActions returns empty
• Hidden information policy (which fields are nulled in getObservation)
• Numeric caps that guarantee finite game length

If any item above is unspecified, do not start engine implementation — return to rule design first.

Part 2: Quality Measurement Methodology

Once the engine satisfies Part 1, evaluate quality on two layers.

Run Part 2 when quality itself is the question: a new or reworked rule set, an explicit balance or bot-strength assessment, or a change to scoreState, action legality, or bot logic. UI, wording, rendering, and pure serialization changes that leave the rule semantics untouched need the Part 1 contract checks only — do not re-run the ladder and metric suite for them.

Layer A: Win-rate ladder

Implement three bots over the contract:

• Random — uniform sample from getLegalActions
• Baseline — greedy 1-step argmax of scoreState
• Strong — shallow search (1–2 ply lookahead) over scoreState

Measure with N ≥ 200 games per matchup, swapping seats to control for first-move advantage.

Layer B: Experience metrics

Per game, snapshot scoreState each turn. Derive WP (win probability) as WP = sigmoid((score0 − score1) / T). Calibrate temperature T per game so that Random-vs-Strong terminal WP for Strong averages ≈ 0.9.

Failure-pattern checklist

When metrics miss targets, look for:

• Degenerate strong play — Strong wins but uses only 1–2 distinct actions
• Dead mechanic — a specific action never used by any bot (dominated or over-costed)
• Seat-order flip — win/loss reverses entirely when seats swap (first-move balance broken)
• Low-impact loop — long games dominated by repeated near-zero ΔWP turns

Improvement-loop discipline

• Change either rules OR bots in a single experiment, never both. Attribution dies if both move.
• Define pass/fail thresholds before the rerun, not after the numbers come back.
• Keep a per-experiment changelog: hypothesis / changed files / before-after metrics / decision (adopt or revert).