Run Gauntlet Challenge
Present challenges from the knowledge base and evaluate answers.
When NOT To Use
- The knowledge base does not exist yet (use
gauntlet:extract) - A staged path for a new contributor (use
gauntlet:onboard)
Challenge Sources
scripts/challenge_engine.py draws from one of three sources:
--source knowledge(default): an entry from.gauntlet/knowledge.json, weighted by the developer's progress--source bank: a problem from the shipped problem bank, narrowed with--categoryand--difficulty, and varied so its answer cannot be memorised--source graph: an impact or dependency question about a real call edge in.gauntlet/graph.db
The bank variation goes through the in-loop provider inside Claude
Code and the Anthropic API outside it (issue #464). The script calls
register_in_loop_provider_if_inside_claude_code() in its own process,
so no setup step is needed.
Steps
Load state: read
.gauntlet/knowledge.jsonand developer progressCheck for pending challenge: if
.gauntlet/state/pending_challenge.jsonexists, evaluate the developer's most recent message as an answer before generating a new oneGenerate challenge: run
python3 ${CLAUDE_PLUGIN_ROOT}/scripts/challenge_engine.py .gauntletwith the source that fits: the default for codebase knowledge, a--source bank --category <name>for practice problems, or--source graphonce a graph has been builtPresent challenge: show the question with context
Evaluate answer: score the response (pass/partial/fail)
Record result: update developer progress and streak
On pass: when the challenge came from the pre-commit gate, score it with the command below, which writes the one-time token that releases the commit of the staged files. Under
--gatethe script scores.gauntlet/state/pending_challenge.jsonand ignores the file argument. Show next challenge if in session.python3 ${CLAUDE_PLUGIN_ROOT}/scripts/answer_evaluator.py \ .gauntlet/state/pending_challenge.json "<answer>" --gateOn fail: show correct answer with explanation. Present a new challenge.
Scoring
| Result | Score | Streak |
|---|---|---|
| Pass | 1.0 | +1 |
| Partial | 0.5 | reset |
| Fail | 0.0 | reset |
Exit Criteria
-
.gauntlet/knowledge.jsonexists and is readable before a challenge is generated; if missing, the skill surfaces the error and suggests runninggauntlet:extract - Each challenge attempt results in a score (1.0/0.5/0.0) written to the developer's progress store and a streak update applied
-
.gauntlet/state/pending_challenge.jsonis evaluated before generating a new challenge when it exists; the file is removed or updated after evaluation - A
--source bankor--source graphrequest yields a challenge from that source, or the script's error naming what is missing - On fail, the correct answer and explanation are shown before the next challenge is presented (not skipped silently)
- On a pass from the pre-commit gate,
.gauntlet/state/pass_token.jsonexists and names the staged set, so the retried commit is released
