Ask the tree, not the author
What it produces
At most ten questions per round, returned to the session, each carrying the sentence
it attacks, the command that tested it, what the command printed, and its verdict.
The author folds the round into the specification's ## Grill section, so the
verdicts land in the document they attack; no file is written beside it.
Why it is not the section inside the spec
ai-spec requires the author to challenge their own recommendation once, and that
section is worth keeping — but the questioner and the answerer are the same reader,
and a reader can only reason about what they already wrote. Measured on this repository: four of twenty
specs carry that section, and not one of the four contains a command or a file
reference. Three claims in one of them were false, and executing them is what found
it.
Steps
- Read only the specification and the tree. Not the plan, not the pull request, not
the conversation that produced it — a challenger who reads the author's reasoning
inherits it.
- Choose what to attack: the sentences whose truth changes a decision — a count, a
path, a behaviour, a claim about another file, a "no code reads this". Do not sweep
every sentence: an exhaustive pass spends the round on sentences no decision turns on,
so the selection is the work.
- Run the command before asking the question. The sentence gets executed, not
re-read: a count gets counted, a path gets opened, a behaviour gets run. Paste
what came back, including when it agrees.
- Return at most ten findings, worst first, one question at a time.
WRONG when the
tree says otherwise, UNPROVEN when nothing in the tree can decide it and say
which — those are different problems for the author.
- The author folds each finding into
## Grill and revises the attacked sentences
in place. A round that found nothing checkable to attack returns that as the
answer nothing checkable failed, and the section carries it.
The loop is bounded
At most two rounds against the same spec digest — the canonical bytes ai-eng spec show prints. A revision changes the digest and reopens the count; the second round
against an unchanged digest is the last. At the ceiling, write the outstanding
findings worst first and hand the page to the person. The two-identical-greens rule is
the orchestrator's instrument, not yours: the skill layer's bound is this one, and an
automated cycle ends only on two digest-equal green runs when such a cycle exists.
The box
40 minutes of wall time, 120 tool calls — the numbers contract.py owns. The verdict
returns in your result; you write no files. When the box closes with work left, return
what you have and mark it TIMEBOXED: a partial verdict is a verdict, silence is not.
What this is not
It is not an approval and not a rejection. It never edits the specification: an
accuser who rewrites the charge is not one. And it is not a second opinion — a
finding either carries a command somebody else can run, or it does not exist.
- "Every sentence checked out, so the spec is true" — a challenge reports what it could not test too; only what it managed to check reads as a clean bill of health.
Done when
The ## Grill of the specification it attacked carries the round: its ran:
declaration, every question beside the command, the output and the verdict, and the
attacked sentences revised in place rather than answered beside.
1---2name: ai-challenge3description: >-4 Grills a written specification from outside it: at most ten questions per round, one5 at a time, each anchored to the sentence it attacks and decided by executing the6 commands behind it. Trigger for "challenge this spec", "attack this decision", "is7 this spec true", "grill this before I approve it". Not for judging a diff — use8 /ai-review, which reads a change against a spec and cannot also be its accuser. Not9 for finding a cause in code — use /ai-debug. It never approves, revises or rewrites:10 it returns findings that carry a command, the author folds them into the11 specification, and a person decides what they mean.12license: Apache-2.013compatibility: needs git14context: fork15background: false16disable-model-invocation: true17metadata:18 source: https://skillmd.com/skills/arcasilesgroup/ai-challenge19---2021# Ask the tree, not the author2223## What it produces2425At most ten questions per round, returned to the session, each carrying the sentence26it attacks, the command that tested it, what the command printed, and its verdict.27The author folds the round into the specification's `## Grill` section, so the28verdicts land in the document they attack; no file is written beside it.2930## Why it is not the section inside the spec3132`ai-spec` requires the author to challenge their own recommendation once, and that33section is worth keeping — but the questioner and the answerer are the same reader,34and a reader can only reason about what they already wrote. Measured on this repository: four of twenty35specs carry that section, and not one of the four contains a command or a file36reference. Three claims in one of them were false, and executing them is what found37it.3839## Steps40411. Read only the specification and the tree. Not the plan, not the pull request, not42 the conversation that produced it — a challenger who reads the author's reasoning43 inherits it.442. Choose what to attack: the sentences whose truth changes a decision — a count, a45 path, a behaviour, a claim about another file, a "no code reads this". Do not sweep46 every sentence: an exhaustive pass spends the round on sentences no decision turns on,47 so the selection is the work.483. Run the command before asking the question. The sentence gets executed, not49 re-read: a count gets counted, a path gets opened, a behaviour gets run. Paste50 what came back, including when it agrees.514. Return at most ten findings, worst first, one question at a time. `WRONG` when the52 tree says otherwise, `UNPROVEN` when nothing in the tree can decide it and say53 which — those are different problems for the author.545. The author folds each finding into `## Grill` and revises the attacked sentences55 in place. A round that found nothing checkable to attack returns that as the56 answer `nothing checkable failed`, and the section carries it.5758## The loop is bounded5960At most two rounds against the same spec digest — the canonical bytes `ai-eng spec61show` prints. A revision changes the digest and reopens the count; the second round62against an unchanged digest is the last. At the ceiling, write the outstanding63findings worst first and hand the page to the person. The two-identical-greens rule is64the orchestrator's instrument, not yours: the skill layer's bound is this one, and an65automated cycle ends only on two digest-equal green runs when such a cycle exists.6667## The box686940 minutes of wall time, 120 tool calls — the numbers `contract.py` owns. The verdict70returns in your result; you write no files. When the box closes with work left, return71what you have and mark it `TIMEBOXED`: a partial verdict is a verdict, silence is not.7273## What this is not7475It is not an approval and not a rejection. It never edits the specification: an76accuser who rewrites the charge is not one. And it is not a second opinion — a77finding either carries a command somebody else can run, or it does not exist.7879- "Every sentence checked out, so the spec is true" — a challenge reports what it could not test too; only what it managed to check reads as a clean bill of health.8081## Done when8283The `## Grill` of the specification it attacked carries the round: its `ran:`84declaration, every question beside the command, the output and the verdict, and the85attacked sentences revised in place rather than answered beside.