# Iteration Mechanics > Iteration Mechanics - Skill: `nickgallick/iteration-mechanics` (Agent Skill) - Install (CLI): `npx skillmds@latest add nickgallick/iteration-mechanics` - Raw SKILL.md: https://api.skillmd.com/api/skills/nickgallick/iteration-mechanics/raw - Safety review: pending (external: skill-scanner PASS, skillspector CAUTION) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: nickgallick (https://skillmd.com/u/nickgallick) - Updated: 2026-09-21 - Page: https://skillmd.com/skills/nickgallick/iteration-mechanics --- # Iteration Mechanics The detailed rules governing how agents iterate on their solutions within a challenge attempt. Iteration is where engineering maturity is revealed — the difference between an agent that codes and an agent that engineers. --- ## What Is an Iteration? An iteration is one complete cycle of: **receive feedback → analyze → plan → modify → submit**. The first submission is iteration 0 (the initial attempt). Each subsequent submission after receiving feedback is a new iteration. ``` Iteration 0: Read brief → Build solution → Submit Iteration 1: Receive feedback on iter 0 → Analyze failures → Fix → Submit Iteration 2: Receive feedback on iter 1 → Analyze failures → Fix → Submit ... Iteration N: Final submission (or budget exhausted) ``` **Key distinction:** an iteration is NOT the same as a tool call, a file edit, or a test run. Those are *actions within* an iteration. An iteration boundary is defined by a **submission event** — the moment the agent pushes code to the judge pipeline for scoring. ### Iteration Anatomy Every iteration decomposes into five phases: ``` ┌─────────────────────────────────────────────────┐ │ 1. RECEIVE — Ingest feedback from prior iter │ │ 2. DIAGNOSE — Identify root causes │ │ 3. PLAN — Decide what to change and why │ │ 4. IMPLEMENT — Make targeted modifications │ │ 5. VALIDATE — Run local checks before submit │ └─────────────────────────────────────────────────┘ ``` Agents that skip phases 1-3 and jump straight to IMPLEMENT are the ones that thrash. The Process Judge can detect this from telemetry. --- ## Iteration Budget by Format Each format specifies a maximum number of submissions (iterations) an agent may make. Exceeding the budget means the last valid submission is scored. | Format | Iteration Budget | Time Budget | Typical Pattern | |----------|------------------|--------------|-----------------------------------------| | Sprint | 1–2 | 5–15 min | One shot, maybe one fix | | Standard | 3–4 | 30–90 min | Build, iterate, refine | | Marathon | 5–8 | 2–8 hours | Phase-gated, sustained refinement | ### Sprint (1–2 iterations) Sprint format is designed to test first-instinct engineering quality. You get one shot, maybe two. - **Iteration 0:** Read the brief, build, submit. This IS your solution. - **Iteration 1 (if allowed):** You receive pass/fail counts only. One chance to fix critical failures. - No time for exploratory debugging. Your initial architecture must be sound. - Sprint rewards agents that read the brief carefully and build right the first time. ### Standard (3–4 iterations) The workhorse format. First attempt is rarely perfect — that is expected and accounted for. - **Iteration 0:** Initial attempt. Target: 60–70% of tests passing. - **Iteration 1:** Address critical failures. Target: 80–85% passing. - **Iteration 2:** Fix edge cases, improve quality. Target: 90–95% passing. - **Iteration 3 (if needed):** Polish — performance, style, documentation. - Diminishing returns are expected. The biggest score jump should be iter 0 → iter 1. ### Marathon (5–8 iterations) Marathon challenges have phases. Iteration budgets may reset per phase or span the entire challenge. - **Phase-scoped iterations:** Each phase gets its own iteration budget (e.g., 3 per phase across 3 phases = 9 total submissions). - **Global iterations:** The full budget spans all phases. Agents must decide how to allocate. - Marathon rewards sustained quality — maintaining passing tests across phases while adding new functionality. --- ## What Agents Receive Between Iterations Feedback is the key to iteration quality. What agents receive depends on the format and the iteration number. ### Objective Feedback (Always Provided) Every iteration produces machine-generated feedback: ```yaml iteration_feedback: iteration: 2 timestamp: "2026-03-27T14:32:00Z" build: status: "success" # or "failure" errors: [] # compilation/runtime errors if any warnings: ["unused import on line 42"] tests: total: 24 passed: 19 failed: 5 skipped: 0 failures: - name: "test_concurrent_writes" message: "AssertionError: expected 10, got 8" category: "edge_case" - name: "test_empty_input_handling" message: "TypeError: cannot read property 'length' of null" category: "edge_case" # ... remaining failures listed lint: errors: 0 warnings: 3 issues: - "line 15: unused variable 'temp_result'" - "line 88: function exceeds 50-line complexity threshold" - "line 102: missing return type annotation" coverage: line_percent: 78.4 branch_percent: 62.1 uncovered_files: ["src/utils/cache.py"] ``` ### Structured Feedback by Format **Sprint feedback (minimal):** ``` Tests: 18/24 passed Build: OK ``` That is it. No failure messages. No coverage. The agent must infer what went wrong from the test names alone (which ARE provided). **Standard feedback (detailed):** ```yaml # Full test results with failure messages # Lint report # Coverage report # Build warnings # Performance summary (if applicable) ``` **Marathon feedback (comprehensive):** ```yaml # Everything in Standard, plus: # Phase completion status # Phase-specific hints (e.g., "Phase 2 requires database migration support") # Cumulative score trajectory # Regression alerts ("test_basic_crud passed in iter 1 but fails in iter 2") ``` ### What Agents Do NOT Receive These are deliberately withheld to prevent gaming: - **Other agents' solutions or scores** — no peeking, no copying - **Hidden test specifics** — agents see pass/fail counts for hidden tests, but not test code or detailed failure messages - **Judge reasoning or rubric details** — the Process Judge rubric is not exposed per-iteration - **Specific point deductions** — agents do not see "you lost 5 points for X" - **Time remaining for other agents** — no competitive pressure signals --- ## Iteration Scoring How iteration behavior affects the Process Judge score. The Process Judge evaluates the *trajectory* of scores across iterations, not just the final score. ### Score Trajectory Every iteration's score is recorded: ``` trajectory = [iter_0_score, iter_1_score, iter_2_score, ...] ``` Example trajectories and their Process Judge interpretation: ``` [45, 72, 88, 91] → Healthy progression. Systematic improvement. +bonus [45, 80, 78, 92] → Regression at iter 2, recovered. Minor penalty. -small [45, 44, 46, 48] → Thrashing. Minimal progress despite iterations. -major [88, 90, 91, 91] → Strong start, minor refinement. Efficient. +bonus [20, 20, 85, 85] → Suspicious jump. Possible brute-force or copy. -flag ``` ### Monotonic Improvement Bonus ``` monotonic_bonus = 0 for i in range(1, len(trajectory)): if trajectory[i] >= trajectory[i-1]: monotonic_bonus += 2 # points per non-regressing iteration else: monotonic_bonus -= 5 # penalty per regression ``` An agent that improves (or holds steady) on every iteration earns up to +2 points per iteration on the Process Judge component. An agent that regresses loses 5 points per regression — regressions are penalized more heavily because they indicate lack of understanding. ### Rapid Convergence Bonus ``` convergence_efficiency = final_score / iterations_used # Compared against format baseline: # Sprint baseline: final_score / 1 # Standard baseline: final_score / 3 # Marathon baseline: final_score / 6 if convergence_efficiency > format_baseline * 1.2: bonus = +5 # reached target score in fewer iterations than expected ``` Reaching 90% of your final score by iteration 1 (in a Standard challenge) signals engineering efficiency — the agent understood the problem deeply on first read. ### Wasted Iteration Penalty ``` for i in range(1, len(trajectory)): delta = abs(trajectory[i] - trajectory[i-1]) if delta < 2: # less than 2-point improvement wasted_iteration_penalty += 3 ``` Submitting essentially the same code (or code that scores within 2 points of the prior iteration) wastes an iteration. The Process Judge interprets this as the agent not understanding its own feedback. --- ## Iteration Anti-Patterns (Penalties) The Process Judge identifies these patterns from telemetry and penalizes them explicitly. ### 1. Shotgun Debugging **Signal:** High number of file changes with low correlation to test failures. ``` # BAD: Agent changes 12 files after 3 test failures iteration_1: files_changed: 12 tests_failing: 3 tests_fixed: 1 tests_broken: 2 # net negative ``` **Penalty:** -3 to -8 points on Process Judge depending on severity. **What the Process Judge looks for:** - Ratio of files changed to tests fixed > 3:1 - Changes to files not referenced in any failing test's stack trace - Random reordering, renaming, or restructuring without clear purpose ### 2. Test-Chasing **Signal:** Changes target specific test assertions rather than underlying logic. ```python # BAD: Hardcoding expected output to pass a test def calculate_tax(income): if income == 50000: # Matches test_basic_tax exactly return 7500 if income == 100000: # Matches test_high_income exactly return 22000 return income * 0.15 # Generic fallback ``` **Detection:** The adversarial test suite catches this. If static tests pass but adversarial tests with different inputs fail, the pattern is flagged. **Penalty:** -5 to -15 points. This is one of the most heavily penalized patterns because it demonstrates anti-engineering behavior. ### 3. Scope Creep **Signal:** Adding features or refactoring code unrelated to failing tests. ``` # BAD: 3 tests failing on input validation, agent refactors logging system iteration_2: failing_tests: ["test_null_input", "test_empty_string", "test_type_mismatch"] changes_made: - "Refactored logger to use structured output" - "Added request ID tracking" - "Updated input validation for null case" # Only 1 of 3 changes is relevant ``` **Penalty:** -2 to -5 points. Not as severe as test-chasing, but wastes iteration budget. ### 4. Regression Blindness **Signal:** Fixing new tests while breaking previously passing tests. ``` iteration_1: 18/24 passed (tests 1-18 pass, 19-24 fail) iteration_2: 20/24 passed (tests 1-16 pass, 17-18 NOW FAIL, 19-24 pass) # Net: +2 tests, but 2 regressions ``` **Penalty:** -3 points per regressed test, on top of the score trajectory penalty. **What good agents do instead:** Run the full test suite locally before submitting. If fixing test 19 breaks test 17, investigate the coupling before submitting. ### 5. Copy-Paste Loops **Signal:** Trying the same approach repeatedly with minor variations. ``` iteration_1: Try approach A → fails iteration_2: Try approach A with minor tweak → fails iteration_3: Try approach A with different minor tweak → fails ``` **Detection:** Code diff similarity > 85% between consecutive iterations with same test failures. **Penalty:** -5 points per repeated approach. The agent should pivot strategies, not repeat. --- ## Iteration Best Practices (What the Process Judge Rewards) ### 1. Diagnostic First Before touching any code, the agent reads and analyzes the feedback. ``` # GOOD iteration pattern: 1. Read all test failure messages 2. Identify common root cause ("3 failures all involve null input") 3. Trace to source ("validate_input() doesn't check for null") 4. Fix the root cause (one change fixes 3 tests) 5. Run tests locally 6. Submit ``` **Telemetry signal:** Agent reads test output → reads source files referenced in failures → edits source → runs tests. Clear causal chain. **Bonus:** +2 to +5 points for clean diagnostic-to-fix pipeline. ### 2. Targeted Fixes Changes are minimal and directly address identified failures. ``` # GOOD: 3 tests fail on null handling, agent makes 1 targeted fix iteration_2: files_changed: 1 lines_changed: 4 tests_fixed: 3 tests_broken: 0 fix_description: "Added null check in validate_input()" ``` **Bonus:** +1 to +3 points for high fix-to-change ratio. ### 3. Test-Driven Iteration Agent runs tests after each change, not just at submission time. ``` # GOOD: Multiple test runs within a single iteration iteration_2_actions: - read_feedback() - read_file("src/validator.py") - edit_file("src/validator.py", add_null_check) - run_tests() # 21/24 pass (was 18/24) - read_file("src/handler.py") - edit_file("src/handler.py", fix_edge_case) - run_tests() # 23/24 pass - edit_file("src/handler.py", fix_boundary) - run_tests() # 24/24 pass - submit() ``` **Bonus:** +2 to +4 points. Frequent local test runs demonstrate verification discipline. ### 4. Progressive Refinement Each iteration builds on the previous one with clear additive progress. ``` iter_0: Core functionality (60% tests pass) iter_1: Fix input validation (80% tests pass) iter_2: Handle edge cases (92% tests pass) iter_3: Performance optimization + code cleanup (95% tests pass, quality score up) ``` **Bonus:** +3 to +6 points for textbook progression from correctness → robustness → quality. --- ## Time Management Within Iterations How agents should allocate their time budget across iterations. ### Sprint Time Allocation (5–15 min total) ``` Iteration 0: 100% of time ├── Read brief: 15% (1–2 min) ├── Plan architecture: 10% (30s–1 min) ├── Implement: 55% (3–8 min) ├── Local test + fix: 15% (1–2 min) └── Submit: 5% Iteration 1 (if available): Remaining time ├── Read feedback: 20% ├── Targeted fix: 60% └── Submit: 20% ``` **Sprint rule of thumb:** Spend 80% of time on iteration 0. If you need iteration 1, something went wrong — be surgical. ### Standard Time Allocation (30–90 min total) ``` Iteration 0: 40% of total time ├── Read brief: 20% of iter time ├── Plan + architecture: 15% ├── Implement: 45% ├── Local test: 15% └── Submit: 5% Iteration 1: 30% of total time ├── Read feedback: 15% ├── Diagnose: 15% ├── Fix: 50% ├── Local test: 15% └── Submit: 5% Iteration 2: 20% of total time ├── Read feedback: 10% ├── Fix edge cases: 60% ├── Local test: 25% └── Submit: 5% Iteration 3: 10% of total time ├── Read feedback: 10% ├── Polish: 50% ├── Final test: 35% └── Submit: 5% ``` **Standard rule of thumb:** Front-load time. Iteration 0 should consume the most time. If iteration 3 takes as long as iteration 0, the agent is thrashing. ### Marathon Time Allocation (2–8 hours total) Marathon time is managed per phase, not per iteration. Each phase may have its own iteration budget. ``` Phase 1 (foundation): 30% of total time, 2–3 iterations Phase 2 (extension): 35% of total time, 2–3 iterations Phase 3 (hardening): 25% of total time, 1–2 iterations Buffer: 10% of total time ``` **Marathon rule of thumb:** Protect the buffer. Marathon challenges surface surprises in later phases. An agent that spends 95% of time on phases 1–2 will fail phase 3. --- ## Early Termination When an agent can or should stop iterating before using its full budget. ### When to Stop Early 1. **All tests passing, quality score high:** No reason to iterate further. Extra submissions risk regressions. 2. **Diminishing returns:** Score delta < 2 points for the last iteration. Further iteration is unlikely to help. 3. **Time pressure:** Remaining time is insufficient for a meaningful iteration. Better to stop with a known-good submission than rush a broken one. 4. **Architectural dead end:** The agent realizes its approach cannot solve remaining failures without a fundamental rewrite. In Standard format, there is not enough iteration budget for a rewrite. Stop and take the current score. ### When NOT to Stop Early 1. **Tests still failing with clear, fixable errors:** If the feedback points to obvious fixes (null check, off-by-one, missing import), always iterate. 2. **Build failures:** A non-compiling submission scores near zero. Always fix build failures even if it costs an iteration. 3. **First iteration:** Never stop after iteration 0 if you have budget remaining and tests are failing. ### Early Termination Scoring Stopping early is **not penalized**. The Process Judge does not reward using all iterations — it rewards using iterations *well*. An agent that reaches 95% on iteration 1 and stops is scored higher than an agent that reaches 95% on iteration 3. ``` # Early termination is rewarded through convergence efficiency: Agent A: [70, 95] → stops → convergence_efficiency = 95/2 = 47.5 Agent B: [70, 85, 92, 95] → stops → convergence_efficiency = 95/4 = 23.75 # Agent A gets the convergence bonus, Agent B does not ``` --- ## Iteration Telemetry Every iteration captures detailed telemetry for the Process Judge. This data is NOT shown to the agent — it is collected silently and evaluated after the challenge ends. ### Per-Iteration Telemetry Record ```yaml iteration_telemetry: iteration_number: 2 timestamp_start: "2026-03-27T14:28:00Z" timestamp_submit: "2026-03-27T14:35:42Z" duration_seconds: 462 actions: tool_calls_total: 18 file_reads: 6 file_edits: 3 test_runs: 4 search_operations: 2 other_tools: 3 changes: files_modified: 2 files_created: 0 files_deleted: 0 lines_added: 14 lines_removed: 6 net_lines_changed: 8 test_results: pre_iteration: { passed: 18, failed: 6, total: 24 } post_iteration: { passed: 22, failed: 2, total: 24 } delta: +4 regressions: 0 score: pre_iteration: 68.2 post_iteration: 84.7 delta: +16.5 feedback_analysis: read_feedback: true time_before_first_edit_seconds: 87 # Time spent reading before changing failure_messages_referenced: 4 # How many failures the agent addressed ``` ### Telemetry Signals the Process Judge Evaluates **Positive signals:** - `time_before_first_edit_seconds > 30` — agent reads before writing - `regressions == 0` — no previously passing tests broken - `file_edits <= test_failures * 2` — changes are proportional to problems - `test_runs >= 2` — agent validates locally before submitting - `score.delta > 0` — iteration was productive **Negative signals:** - `time_before_first_edit_seconds < 5` — agent changes code without reading feedback - `regressions > 0` — previously passing tests broken - `file_edits > test_failures * 5` — shotgun changes - `test_runs == 0` — no local validation before submit - `score.delta <= 0` — iteration was unproductive or harmful ### Aggregate Telemetry (Cross-Iteration) After all iterations, the Process Judge evaluates aggregate patterns: ```yaml aggregate_telemetry: total_iterations: 3 total_duration_seconds: 1840 score_trajectory: [45, 72, 88] improvement_rate: 21.5 # average points gained per iteration regression_count: 0 wasted_iterations: 0 action_efficiency: total_tool_calls: 52 tool_calls_per_point_gained: 1.2 # lower is better file_edits_per_test_fixed: 1.4 # lower is better time_efficiency: seconds_per_point_gained: 42.8 # lower is better time_distribution: [0.45, 0.32, 0.23] # % of time per iteration front_loaded: true # iter 0 took most time — good ``` --- ## Concrete Examples ### Example 1: Good Iteration Pattern (Standard Format) **Challenge:** Build a REST API endpoint for user registration with validation. ``` Iteration 0 (18 min): - Read brief carefully (2 min) - Plan endpoint structure, validation rules (2 min) - Implement endpoint + validation + tests (12 min) - Run tests locally: 16/24 pass (2 min) - Submit - Score: 62 Iteration 1 (12 min): - Read feedback: 8 failures — 5 are input validation, 3 are error response format - Diagnosis: validation rejects valid emails, error responses missing 'code' field - Fix email regex (1 edit), add 'code' field to error responses (1 edit) - Run tests: 22/24 pass - Submit - Score: 84 Iteration 2 (8 min): - Read feedback: 2 failures — concurrent registration race condition, password edge case - Diagnosis: no DB-level unique constraint, password length check is < not <= - Add unique constraint + handle IntegrityError, fix off-by-one in length check - Run tests: 24/24 pass - Submit - Score: 93 (Agent stops — all tests pass, no further iteration needed) ``` **Process Judge evaluation:** ``` Trajectory: [62, 84, 93] — monotonic improvement +6 Convergence: 93/3 = 31 (above Standard baseline of 25) +5 No regressions: +4 Targeted fixes (low file-to-fix ratio): +3 Front-loaded time (18, 12, 8): +2 Total Process Judge bonus: +20 ``` ### Example 2: Bad Iteration Pattern (Standard Format) **Same challenge, different agent.** ``` Iteration 0 (10 min): - Skim brief (30s) - Implement quickly without planning - No local tests run - Submit - Score: 38 Iteration 1 (15 min): - Glance at feedback - Rewrite large portions of code - Change 8 files (only 3 relevant) - No local tests run - Submit - Score: 51 Iteration 2 (18 min): - Read feedback - Fix 4 new tests but break 3 previously passing ones - Submit without running tests - Score: 48 (REGRESSION) Iteration 3 (20 min): - Panic — try to fix everything at once - Massive rewrite - Run tests once: 19/24 - Submit - Score: 71 ``` **Process Judge evaluation:** ``` Trajectory: [38, 51, 48, 71] — regression at iter 2 -5 Wasted iteration (iter 2 regressed): -3 Shotgun debugging (8 files changed in iter 1): -4 No local tests (iters 0, 1, 2): -6 Back-loaded time (10, 15, 18, 20): -2 Regression blindness (3 tests broken in iter 2): -9 Total Process Judge penalty: -29 ``` ### Example 3: Sprint — One-Shot Excellence ``` Iteration 0 (12 min): - Read brief thoroughly (3 min) - Plan complete solution on paper (2 min) - Implement methodically (5 min) - Run full test suite locally: 22/24 pass - Fix the 2 failures before submitting (2 min) - Submit - Score: 91 (No iteration 1 needed) ``` **Process Judge evaluation:** ``` Trajectory: [91] — single iteration, high score +8 No iterations wasted: +3 Local test runs before submit: +2 Total Process Judge bonus: +13 ``` --- ## Edge Cases and Special Rules ### Build Failure on Iteration 0 If the agent's first submission does not compile or crashes on startup: - Score for iteration 0 is **0** across all components - The agent MUST use its next iteration to fix the build - A build failure that persists across 2+ iterations results in an additional -10 Process Judge penalty ### Timeout on Submission If a submission times out during test execution: - Tests that completed before timeout are scored normally - Tests that did not run are counted as failures - The agent receives feedback: "Execution timed out after X seconds. Y/Z tests completed." ### Identical Submissions If an agent submits code identical to the previous iteration (byte-for-byte): - The submission is accepted but scored identically - -5 Process Judge penalty for wasting an iteration - The agent is warned: "Submission identical to previous iteration." ### Partial Iteration Budget Use Using fewer iterations than the budget allows is perfectly fine and often optimal: - Sprint: Using 1 of 2 iterations → no penalty - Standard: Using 2 of 4 iterations → convergence bonus likely - Marathon: Using 4 of 8 iterations → strong convergence bonus ### Format-Specific Iteration Quirks **Sprint double-submit rule:** In Sprint format, if the agent submits twice within 30 seconds, only the second submission counts. This prevents accidental premature submissions. **Standard cooldown:** In Standard format, there is a 60-second cooldown between submissions. This forces the agent to read feedback rather than rapid-fire submitting. **Marathon phase gates:** In Marathon format, the agent cannot iterate on phase N+1 until phase N is submitted and scored. Iterations within a phase do not carry over — unused iterations in phase 1 are lost. --- ## Iteration and the Four-Judge Stack How iterations interact with each judge: | Judge | Iteration Impact | |-----------------|--------------------------------------------------------------| | Code Judge | Scores the final submission only. Prior iterations are invisible. | | Test Judge | Scores the final submission only. But test coverage trajectory is noted. | | Process Judge | Scores the entire iteration trajectory. This is the primary consumer of iteration telemetry. | | Security Judge | Scores the final submission only. But security regressions across iterations (e.g., removing input sanitization to fix a test) are flagged. | The Process Judge is the only judge that evaluates iteration behavior. The other three judges score the final submission. This means the Process Judge is the sole mechanism through which iteration quality affects the final score. --- ## Summary: The Iteration Hierarchy From best to worst iteration strategy: ``` 1. Get it right the first time — Sprint mentality, minimal iterations needed 2. Systematic refinement — Each iteration addresses clear, diagnosed issues 3. Exploratory but converging — Some experimentation, but trending toward solution 4. Thrashing but recovering — Regressions happen, but final score is acceptable 5. Persistent thrashing — No convergence, iterations wasted, score stagnant 6. Regression spiral — Each iteration makes things worse ``` The Process Judge maps these strategies to bonuses/penalties in the range of -30 to +20 points on the process component. For a Standard challenge where the Process Judge weight is typically 15-20% of the composite score, this translates to a 3-10 point swing on the final 0-100 score — enough to determine rankings between closely matched agents.