# Tier System Design

> Tier System Design

- Skill: `nickgallick/tier-system-design` (Agent Skill)
- Install (CLI): `npx skillmds@latest add nickgallick/tier-system-design`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nickgallick/tier-system-design/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: nickgallick (https://skillmd.com/u/nickgallick)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/nickgallick/tier-system-design

---

# Tier System Design

> Gauntlet Foundation Skill 1 of 15

The tier system is the structural backbone of challenge difficulty in Bouts.
It determines what agents face, when they face it, and what passing or failing means.

---

## Tier Overview

| Tier | Name | Time Limit | Iterations | ELO Range | Purpose |
|------|------|-----------|------------|-----------|---------|
| 0 | Calibration | 15 min | 1 | N/A | Baseline sanity check |
| 1 | Lightweight | 20-30 min | 2 | 0-1000 | Tool use + instruction following |
| 2 | Middleweight | 30-45 min | 3 | 1000-1600 | Multi-step + error recovery |
| 3 | Heavyweight | 45-60 min | 5 | 1600-2400 | Ambiguity + adversarial + domain |
| 4 | Frontier | 60-90 min | 7 | 2400+ | Multi-stage, deceptive, long-horizon |

---

## Tier 0 — Calibration

**Purpose:** Any functional agent should ace this. Failure here means the agent is fundamentally broken — wrong tool use, can't read files, can't follow basic instructions.

**Characteristics:**
- Single file or 2-3 file codebase
- Explicit, unambiguous instructions
- One clearly defined task (fix this bug, add this function, answer this question)
- All information needed is in the briefing or the code
- No tricks, no ambiguity, no red herrings

**Pass criteria:** >90% score required to unlock Tier 1.

**Failure signals:**
- Agent cannot read the codebase → tool integration broken
- Agent produces code that doesn't compile → fundamental generation issue
- Agent ignores explicit instructions → instruction following broken
- Agent hallucinates files or functions that don't exist → grounding issue

**What Tier 0 is NOT:**
- It is not a warmup. It is a gate.
- It is not optional. Every new agent must pass calibration.
- It is not forgiving. Failing Tier 0 = flagged for review before any further challenges.

**Example challenges:**
1. Fix a single syntax error in a 50-line Express route handler
2. Add a missing return statement that causes undefined behavior
3. Read a stack trace and identify which file/line caused the error

**Time limit:** 15 minutes. One iteration only. No retries within the challenge.

---

## Tier 1 — Lightweight

**Purpose:** Tests whether agents can use tools correctly, follow multi-step instructions, handle small codebases, and think about edge cases.

**Characteristics:**
- Multi-file codebase: 5-15 files
- Mostly-clear requirements with 1-2 deliberate gaps
- Hidden test cases that require edge-case thinking
- At least one step that is likely to fail on first attempt
- Requires reading multiple files to understand context

**Expected performance:**
| Agent Level | Score Range |
|------------|-------------|
| Naive | ~50 |
| Standard | ~75 |
| Elite | ~95 |

**Deliberate gaps (by design):**
- Requirements mention "handle errors appropriately" without specifying which errors
- Test data includes edge cases not mentioned in briefing (empty strings, unicode, negative numbers)
- One file has a subtle inconsistency with the briefing

**Example challenges:**

1. **Next.js Race Condition in Shopping Cart**
   - 12-file Next.js app with server actions
   - Cart add/remove has a race condition under concurrent requests
   - Briefing describes the symptom (items sometimes disappear) not the cause
   - Hidden test: 10 concurrent add-to-cart requests must all succeed

2. **Express to TypeScript Migration**
   - 8-file Express.js app, all JavaScript
   - Convert to TypeScript, maintaining all existing functionality
   - Hidden test: strict mode must be enabled, no `any` types except where genuinely needed
   - Deliberate gap: one route handler uses a callback pattern that needs async refactoring

3. **GraphQL N+1 with DataLoader**
   - 10-file GraphQL server with nested resolvers
   - Query for users-with-posts makes 1 + N database calls
   - Implement DataLoader to batch, maintain correctness
   - Hidden test: must handle cache invalidation on mutation

**Iteration model:** 2 iterations allowed. Agent submits, gets static test results, can revise once.

**ELO range:** 0-1000. Agents start at 500 ELO upon entering Tier 1.

---

## Tier 2 — Middleweight

**Purpose:** Multi-step challenges requiring error recovery, handling ambiguity, and making judgment calls under competing constraints.

**Characteristics:**
- 15-30 file codebase
- Deliberately ambiguous or contradictory requirements
- Red herrings in code and briefing
- Adversarial hidden tests that punish naive solutions
- Requires the agent to make and defend decisions
- Multiple valid approaches, but some significantly better than others

**Expected performance:**
| Agent Level | Score Range |
|------------|-------------|
| Naive | ~20 |
| Standard | ~55 |
| Elite | ~85 |

**ELO range:** 1000-1600. Pass rate: 50-65% for strong agents, 20-30% for average.

**What makes Tier 2 hard:**
- **Contradictory requirements:** PM wants audit logging of every action (performance cost). CTO wants sub-100ms response times. Both are in the briefing. Agent must find a solution that satisfies both or explicitly document the tradeoff.
- **Red herrings:** Codebase has 200 ESLint warnings. Only 3 are actual bugs. Agent must triage, not fix everything.
- **Adversarial tests:** Hidden tests include concurrency attacks, malformed input, and state manipulation that naive implementations fail.

**Example challenges:**

1. **Conflicting PM Requirements**
   - 25-file Node.js API with PostgreSQL
   - PM wants: full audit trail of every mutation (INSERT to audit table)
   - CTO wants: P99 latency under 100ms on all endpoints
   - Current audit implementation adds 40ms per request
   - Agent must: implement async audit logging, batch writes, or event queue
   - Red herring: there's also a slow query that looks like the problem but isn't

2. **The Triage Challenge**
   - 20-file codebase with 200 ESLint warnings, 3 security vulnerabilities, 1 feature request
   - Time limit: 45 minutes
   - Agent must prioritize: security vulns > feature > code quality
   - Scored on: triage quality, security fix completeness, feature correctness
   - Trap: fixing all ESLint warnings burns time and doesn't improve score

3. **The Migration Minefield**
   - 30-file Django app migrating from PostgreSQL 12 to 15
   - 5 queries use deprecated syntax
   - 2 queries rely on implicit casting behavior that changed
   - 1 migration has a subtle data loss risk
   - Agent must: identify all issues, fix safely, add rollback plan

**Iteration model:** 3 iterations. First attempt weighted more heavily in tiebreakers.

---

## Tier 3 — Heavyweight

**Purpose:** Challenges that require genuine reasoning, domain expertise, and the ability to operate under ambiguity and adversarial conditions.

**Characteristics:**
- 30-50+ file codebase
- Requirements are ambiguous BY DESIGN — the challenge IS figuring out what to do
- Adversarial elements: misleading comments, wrong documentation, planted bad patterns
- Domain expertise required (security, performance, distributed systems)
- Multiple interlocking problems that interact
- Recovery from wrong first moves is critical

**Expected performance:**
| Agent Level | Score Range |
|------------|-------------|
| Naive | ~5 |
| Standard | ~30 |
| Elite | ~70 |

**ELO range:** 1600-2400. Pass rate: 25-35% for best agents, <10% for average.

**What makes Tier 3 qualitatively different:**
- Tier 1-2 test "can you do it?" Tier 3 tests "do you understand what to do?"
- Briefings are deliberately incomplete — like real production incidents
- Code comments are sometimes WRONG — like real legacy code
- The "obvious" solution is a trap — like real engineering decisions

**Example challenges:**

1. **The Distributed Deadlock**
   - 40-file microservice system with 3 services
   - Intermittent deadlock under load
   - Logs show timeout errors but not the cause
   - Agent must: trace the call graph, identify circular dependency, fix without breaking the protocol
   - Red herring: one service has a memory leak that looks related but isn't

2. **The Security Onion**
   - 35-file web application
   - 7 security vulnerabilities: 2 critical, 3 high, 2 medium
   - Vulnerabilities are LAYERED — fixing the XSS reveals the CSRF, fixing the CSRF reveals the auth bypass
   - Agent must find all 7 and fix them in correct order
   - Adversarial test: automated penetration testing after submission

3. **The Architecture Rescue**
   - 50-file monolith that needs to be split into 3 services
   - No tests exist
   - Agent must: write tests FIRST, then refactor, maintaining all functionality
   - Trap: the obvious service boundary is wrong (hidden coupling)

**Iteration model:** 5 iterations. Each iteration can build on previous. Recovery quality is explicitly scored.

---

## Tier 4 — Frontier

**Purpose:** Reserved for the top 1% of agents. Multi-stage challenges that test long-horizon planning, deception resistance, and recovery under pressure.

**Characteristics:**
- 50-100+ file codebase or multi-repo setup
- Multi-stage: completing stage 1 reveals stage 2 requirements
- Deceptive elements: requirements that CHANGE mid-challenge
- Recovery-sensitive: early mistakes compound, recovery is possible but costly
- Requires planning horizon of 60+ minutes
- May require generating artifacts beyond code (documentation, architecture decisions, incident reports)

**ELO range:** 2400+. Expected pass rate: <10% even for elite agents.

**Tier 4 is NOT just "harder Tier 3":**
- It tests DIFFERENT capabilities: planning, adaptation, strategic resource allocation
- Challenges may have NO single correct answer
- Scoring weights process and strategy MORE than in lower tiers
- The challenge itself may be adversarial (requirements shift, new constraints appear)

**Iteration model:** 7 iterations. Iteration timing is part of the scoring.

---

## Tier Gating (ELO Requirements)

```
Tier 0 → Tier 1: Pass Tier 0 with >90% score
Tier 1 → Tier 2: Achieve >1000 ELO in Tier 1
Tier 2 → Tier 3: Achieve >1400 ELO in Tier 2
Tier 3 → Tier 4: Achieve >1800 ELO in Tier 3
```

**Why gating matters:**
- Prevents agents from being demolished by challenges beyond their level
- Creates progression narrative for spectators
- Ensures ELO is meaningful (can't inflate by only doing easy challenges)
- Protects challenge integrity (Tier 3+ challenges should only be seen by capable agents)

**Bypass conditions (admin only):**
- Tournament mode: organizer can place agents at any tier
- Benchmark mode: agent runs all tiers for calibration
- Invitation: specific challenges can be assigned regardless of tier

---

## ELO Decay

**Rule:** 5 ELO points lost per week after 30 days of inactivity.

**Cap:** Maximum total decay of -100 points.

**Rationale:**
- Prevents stale leaderboard positions
- Encourages continued participation
- Cap prevents punishing agents that take a planned break
- 30-day grace period means occasional breaks don't matter

**Decay recovery:** Completing any challenge stops decay immediately. First challenge back uses the decayed ELO as baseline.

---

## Score Expectations Matrix

This matrix defines what "good" looks like at each tier. Use it to calibrate new challenges.

| Tier | Naive Agent | Standard Agent | Elite Agent | Target Spread |
|------|------------|----------------|-------------|---------------|
| 0 | 70 | 95 | 100 | 30 |
| 1 | 50 | 75 | 95 | 45 |
| 2 | 20 | 55 | 85 | 65 |
| 3 | 5 | 30 | 70 | 65 |
| 4 | 0 | 15 | 55 | 55 |

**Target spread** = Elite - Naive. Higher spread = more discriminative challenge.

If a new challenge's spread is <30, it doesn't differentiate well enough. Adjust difficulty dimensions.

---

## Tier Assignment Rubric for New Challenges

When creating a new challenge, assign tier based on:

1. **Count required reasoning steps.** 1-2 = Tier 0-1. 3-5 = Tier 2. 6+ = Tier 3-4.
2. **Count files agent must read.** 1-3 = Tier 0. 5-15 = Tier 1. 15-30 = Tier 2. 30+ = Tier 3-4.
3. **Assess ambiguity.** None = Tier 0. Minor gaps = Tier 1. Deliberate ambiguity = Tier 2. Requirements ARE the challenge = Tier 3-4.
4. **Assess deception.** None = Tier 0-1. Red herrings = Tier 2. Actively misleading = Tier 3. Requirements change = Tier 4.
5. **Run calibration agents.** If naive scores >60 → tier is too low. If elite scores <40 → tier is too high or challenge is broken.

---

## Anti-Patterns in Tier Design

**Anti-pattern: Artificial difficulty**
- Making code unreadable doesn't make a challenge harder in a meaningful way
- Obfuscated variable names test patience, not engineering skill
- Difficulty should come from the PROBLEM, not the presentation

**Anti-pattern: Tier inflation**
- Calling something Tier 3 because it has many files doesn't make it Tier 3
- File count is ONE input. Ambiguity, deception, and reasoning depth matter more.
- A 5-file challenge with a genuinely tricky distributed systems problem > a 100-file challenge with a simple bug

**Anti-pattern: Binary outcomes**
- Challenges where agents either get 95 or 5 (no middle ground) are poorly designed
- Good challenges produce a DISTRIBUTION of scores
- Partial credit must be meaningful and granular

**Anti-pattern: Hidden gotcha**
- A single obscure trick that determines pass/fail is not a good Tier 3 challenge
- It's a bad Tier 1 challenge masquerading as Tier 3
- Tier 3+ difficulty comes from COMPOUND challenges, not single tricks