# Refine > Use when asked to refine a draft, iterate on code until cleaner, or polish a README through repeated critique and improvement. - Skill: `tony/refine` (Agent Skill) - Install (CLI): `npx skillmds@latest add tony/refine` - Raw SKILL.md: https://api.skillmd.com/api/skills/tony/refine/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: tony (https://skillmd.com/u/tony) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/tony/refine --- # Weave Refine Iteratively improve an artifact through independent adversarial critique and weaving. Host-native sub-agents are the default; separate model CLIs are available by explicit choice. Each pass judges the alternatives, picks the best, incorporates runner-up strengths, and addresses weaknesses. ## When to Use - You have a draft that needs improvement - You want independent adversarial participants to critique and enhance an artifact - You want iterative refinement where each pass genuinely improves the output - You have output from `/weave:brainstorm` that you want to refine further ## Key Features - Accepts inline text or file paths (auto-detected) - `--workers=subagents|model-clis`: Use host-native sub-agents by default or explicitly select separate model CLIs - `--passes=N` (1-5, default 2): Number of refinement cycles - Each pass: all participants critique and improve, judge picks best, weaves in strengths from runners-up - Full rationale chain showing evolution across passes - Early-stop when no material improvement detected - `--judge=host|round-robin`: Host judges every pass, or rotate judging across participants ## The Refinement Cycle 1. All participants independently critique and improve the artifact 2. Judge scores versions, picks the best, identifies strengths in runners-up 3. Judge weaves a revised version incorporating all improvements 4. Woven version goes back to all participants for another round 5. Repeat until passes exhausted or convergence Iteratively improve an artifact through independent adversarial workers using a judge-weave-distribute cycle. Host-native sub-agents are the default; separate model CLIs are optional. Each pass produces independent critiques and improved versions from all workers, then the host agent judges, picks the best, incorporates strengths from runners-up, and distributes the woven result back for the next pass. This is a **project-read-only** command — no files in your repository are written, edited, or deleted. Session artifacts (worker outputs, judge assessments, woven results) are persisted to `$AI_AIP_ROOT` for post-session inspection; this directory is outside your repository. The artifact comes from `$ARGUMENTS`. If no arguments are provided, ask the user what artifact they want refined. ## Worker selection Before any other unresolved configuration choice or operational step, read `../../references/worker-backends.md`. Resolve `worker_backend` from `--workers=subagents|model-clis` using that reference; if the flag is absent, ask its worker question first. If interactive choice is unavailable, honor its documented headless default. The selected backend governs the whole session: dispatch, retry, judging, refinement, artifacts, session metadata, and presentation. The shared reference adapts provider-named examples across every later phase to that backend. When `worker_backend == subagents`, use only the reference's native sub-agent path. Skip every model-CLI detection, timeout question, timeout resolution, retry, fallback, and dispatch instruction below. Every such instruction below is conditional on `worker_backend == model-clis`. --- ## Phase 1: Gather Context and Detect Input **Goal**: Understand the project, detect the input artifact, and prepare the refinement task. 1. **Read CLAUDE.md / AGENTS.md** if present — project conventions inform better critiques and improvements. 2. **Determine trunk branch** (for context about the project state): ```bash git remote show origin | grep 'HEAD branch' ``` Fall back to `main`, then `master`, if detection fails. 3. **Capture and detect the artifact**: Use `$ARGUMENTS` as the user's input. If `$ARGUMENTS` is empty, ask the user what artifact they want refined. **Input detection**: - If `$ARGUMENTS` (after stripping flags) resolves to an existing file path on disk, read the file contents as the artifact. Record the file path as the artifact source. - Otherwise, treat `$ARGUMENTS` (after stripping flags) as inline text. Record "inline text" as the artifact source. Store the original artifact text for use in all subsequent phases. --- ## Phase 1b: Build Context Packet After Phase 1 context gathering (reading CLAUDE.md, exploring files, capturing the artifact), assemble a structured context bundle that will be included verbatim in ALL model prompts. This ensures every model works from the same information. Write to `$SESSION_DIR/context-packet.md` *(the actual file write happens after Session Directory Initialization in Phase 2 creates `$SESSION_DIR`)*: 1. **Conventions summary** — key rules from CLAUDE.md/AGENTS.md (max 50 lines). Focus on commit format, test patterns, code style, and quality gates relevant to the artifact. 2. **Repo state** — branch, HEAD ref, trunk branch, uncommitted changes summary: ```bash git status --short ``` 3. **Changed files** — branch changes relative to trunk: ```bash git diff --stat origin/...HEAD ``` 4. **Relevant file list** — files matching artifact keywords discovered during Phase 1 exploration. Include paths only, not content. 5. **Key snippets** — critical function signatures, types, test patterns, or API contracts relevant to the artifact (max 200 lines). Prioritize interfaces over implementations. 6. **Known unknowns** — aspects of the artifact that need discovery during execution. List what the models should investigate. **Size limit**: 400 lines total. Prioritize by task relevance. If the packet exceeds 400 lines, truncate the least relevant sections (snippets first, then file list). **Usage in model prompts**: - For the **Claude Task agent**: reference the file path (`$SESSION_DIR/context-packet.md`) — the agent reads it directly - For **Antigravity and GPT sub-agents**: include the context packet content in the agent prompt, which the sub-agent then passes to the external CLI For `refine`, prioritize conventions summary and key snippets related to the artifact's domain. --- ## Phase 2: Configuration and Model Detection ### Step 1: Parse Flags Scan `$ARGUMENTS` for explicit flags anywhere in the text. Flags use `--name=value` syntax and are stripped from the prompt text before sending to models. | Flag | Values | Default | Description | |------|--------|---------|-------------| | `--passes=N` | 1–5 | 2 | Number of refinement passes | | `--timeout=N\|none` | seconds or `none` | 450s | Timeout for external model commands | | `--mode=fast\|balanced\|deep` | mode preset | `balanced` | Execution mode preset | | `--judge=host\|round-robin` | judge mode | `host` | Who judges each pass | | `--no-deslop` | flag | off | Skip the final deslop pass on the woven synthesis | | `--quiet-deslop` | flag | off | Replace the 8-line deslop summary with one line | | `--verbose-deslop` | flag | off | Add tier letter, signature id, confidence per finding | **Mode presets** set default passes and timeout when not explicitly overridden: | Mode | Passes | Timeout multiplier | |------|--------|--------------------| | `fast` | 1 | 0.5× default | | `balanced` | 2 | 1× default | | `deep` | 3 | 1.5× default | **`--judge` flag**: `--judge=host` (default) uses the host agent (Claude) as judge for all passes. `--judge=round-robin` rotates judging across available models: Pass 1 → Claude, Pass 2 → Antigravity, Pass 3 → GPT, Pass 4 → Claude, etc. The rotation includes only models that are available (detected in Step 3). If only one model is available, round-robin degrades to host mode. External model judges produce scores and pick winners; the host agent always weaves. **Backward compatibility**: Legacy trigger words are silently recognized as aliases: - `multipass` (case-insensitive) → `--passes=2` - `x` (N = 2–5, regex `\bx([2-5])\b`) → `--passes=N` - `timeout:` → `--timeout=` - `timeout:none` → `--timeout=none` Legacy triggers are scanned on the first and last line only (to avoid false positives in pasted content). Explicit `--` flags take priority over legacy triggers. Values above 5 for `--passes` are capped at 5 with a note to the user. **Config flags** (used in Step 2): - `pass_count` = parsed pass count from `--passes`, mode preset, or legacy trigger. Null if not provided. - `timeout_value` = parsed timeout from `--timeout`, mode preset, or legacy trigger. Null if not provided. - `judge_mode` = parsed from `--judge` or `"host"` default. ### Step 2: Interactive Configuration **When flags are provided, skip the corresponding question.** When `--passes` is provided, skip the passes question. When `--timeout` is provided, skip the timeout question. If `AskUserQuestion` is unavailable (headless mode via `claude -p`), use `pass_count` value if set, otherwise default to 2 passes. Timeout uses `timeout_value` if set, otherwise the command's default timeout (450s). Use `AskUserQuestion` to prompt the user for any unresolved settings: **Question 1 — Passes** (skipped when `--passes` was provided): - question: "How many refinement passes? Each pass runs a full judge-weave-distribute cycle across all models." - header: "Passes" - When `pass_count` exists (from mode preset or legacy trigger), move the matching option first with "(Recommended)" suffix. Other options follow in ascending order. - When `pass_count` is null, use default ordering: - "2 — two passes (Recommended)" — Two full judge-weave cycles. Good balance of quality and cost. - "1 — single pass" — One round of critique and improvement. Quick but limited refinement. - "3 — three passes" — Three full cycles. Maximum depth, highest token usage. **Question 2 — Timeout** (skipped when `--timeout` was provided): - question: "Timeout for external model commands?" - header: "Timeout" - options: - "Default (450s)" — Use this command's built-in default timeout. - "Quick — 225s" — For fast queries (0.5× default). May timeout on complex tasks. - "Long — 675s" — For complex tasks (1.5× default). Higher wait on failures. - "None" — No timeout. Wait indefinitely for each model. ### Step 3: Detect Available Models **Goal**: Check which AI CLI tools are installed locally. Run these checks in parallel: ```bash command -v agy >/dev/null 2>&1 && echo "agy:available" || echo "agy:missing" ``` ```bash command -v gemini >/dev/null 2>&1 && echo "gemini:available" || echo "gemini:missing" ``` ```bash command -v codex >/dev/null 2>&1 && echo "codex:available" || echo "codex:missing" ``` ```bash command -v agent >/dev/null 2>&1 && echo "agent:available" || echo "agent:missing" ``` #### Model resolution (priority order) | Slot | Priority 1 (native) | Native model | Fallback chain | Agent model | |------|---------------------|--------------|----------------|----------- | | **Claude** | Always available (this agent) | — | — | — | | **Antigravity** | `agy` binary | `Gemini 3.1 Pro (High)` | `gemini -m gemini-3-pro-preview` → `agent --model gemini-3.1-pro` | `gemini-3.1-pro` | | **GPT** | `codex` binary | (default) | `agent --model gpt-5.4-high` | `gpt-5.4-high` | **Resolution logic** for each external slot: 1. Native CLI found → use it 2. Else next CLI in the fallback chain → use it (`agent` slots use the `--model` flag) 3. Else → slot unavailable, note in report The **Antigravity** slot is Google's lane: `agy` (Antigravity) supersedes the standalone `gemini` CLI, which Google retires on 2026-06-18. `agy` has no native read-only mode, so read-only commands isolate it in a disposable git worktree (Repo Guard Layer 1; see `docs/repo-guard-protocol.md`). Report which models will participate and which backend each uses. ### Step 4: Detect Timeout Command ```bash command -v timeout >/dev/null 2>&1 && echo "timeout:available" || { command -v gtimeout >/dev/null 2>&1 && echo "gtimeout:available" || echo "timeout:none"; } ``` On Linux, `timeout` is available by default. On macOS, `gtimeout` is available via GNU coreutils. If neither is found, run external commands without a timeout prefix — time limits will not be enforced. Do not install packages automatically. Store the resolved timeout command (`timeout`, `gtimeout`, or empty) for use in all subsequent CLI invocations. When constructing bash commands, replace `` with the resolved command and `` with the resolved value (from trigger parsing, interactive config, or the command's default). If no timeout command is available, omit the prefix entirely. When `--timeout=none` is configured (via flag or interactive selection), also omit `` and `` entirely — run external commands without any timeout prefix. ### Step 5: Detect Search Tools Run these checks in parallel alongside model detection: ```bash command -v rg >/dev/null 2>&1 && echo "rg:available" || echo "rg:missing" ``` ```bash command -v ag >/dev/null 2>&1 && echo "ag:available" || echo "ag:missing" ``` ```bash command -v fd >/dev/null 2>&1 && echo "fd:available" || echo "fd:missing" ``` Use faster search tools when available for context-packet building: | Tool | Purpose | Fallback | |------|---------|----------| | `rg` (ripgrep) | Content search | `ag` then `grep` | | `ag` (silver searcher) | Content search | `grep` | | `fd` | File discovery | `find` | ### Session Directory Initialization #### Step 1: Resolve storage root ```bash if [ -n "$AI_AIP_ROOT" ]; then AIP_ROOT="$AI_AIP_ROOT" elif [ -n "$XDG_STATE_HOME" ]; then AIP_ROOT="$XDG_STATE_HOME/ai-aip" elif [ "$(uname -s)" = "Darwin" ]; then AIP_ROOT="$HOME/Library/Application Support/ai-aip" else AIP_ROOT="$HOME/.local/state/ai-aip" fi ``` Create a `/tmp/ai-aip` symlink to the resolved root for backward compatibility (if `/tmp/ai-aip` doesn't already exist or isn't already correct): ```bash ln -sfn "$AIP_ROOT" /tmp/ai-aip 2>/dev/null || true ``` #### Step 2: Compute repo identity ```bash REPO_TOPLEVEL="$(git rev-parse --show-toplevel)" ``` ```bash REPO_SLUG="$(basename "$REPO_TOPLEVEL" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9._-]/-/g')" ``` ```bash REPO_ORIGIN="$(git remote get-url origin 2>/dev/null || true)" ``` ```bash if [ -n "$REPO_ORIGIN" ]; then REPO_KEY="${REPO_ORIGIN}|${REPO_SLUG}" else REPO_KEY="$REPO_TOPLEVEL" fi ``` ```bash if command -v sha256sum >/dev/null 2>&1; then REPO_ID="$(printf '%s' "$REPO_KEY" | sha256sum | cut -c1-12)" else REPO_ID="$(printf '%s' "$REPO_KEY" | shasum -a 256 | cut -c1-12)" fi ``` ```bash REPO_DIR="${REPO_SLUG}--${REPO_ID}" ``` #### Step 3: Generate session ID ```bash SESSION_ID="$(date -u '+%Y%m%d-%H%M%SZ')-$$-$(head -c2 /dev/urandom | od -An -tx1 | tr -d ' ')" ``` #### Step 4: Create session directory ```bash SESSION_DIR="$AIP_ROOT/repos/$REPO_DIR/sessions/refine/$SESSION_ID" ``` Create the session directory tree: ```bash mkdir -p -m 700 "$SESSION_DIR/pass-0001/outputs" "$SESSION_DIR/pass-0001/stderr" ``` #### Step 5: Write `repo.json` (if missing) If `$AIP_ROOT/repos/$REPO_DIR/repo.json` does not exist, write it with these contents: ```json { "schema_version": 1, "slug": "", "id": "", "toplevel": "", "origin": "" } ``` #### Step 6: Write `session.json` (atomic replace) Write to `$SESSION_DIR/session.json.tmp`, then `mv session.json.tmp session.json`: ```json { "schema_version": 1, "session_id": "", "command": "refine", "status": "in_progress", "branch": "", "ref": "", "worker_backend": "", "participants": ["", "..."], "executors": {"": ""}, "judge_mode": "", "pass_count": , "completed_passes": 0, "prompt_summary": "", "created_at": "", "updated_at": "" } ``` When `worker_backend == model-clis`, add a `"models"` array containing the resolved model for each participant. Omit `"models"` when `worker_backend == subagents`. #### Step 7: Append `events.jsonl` Append one event line to `$SESSION_DIR/events.jsonl`: ```json {"event":"session_start","timestamp":"","command":"refine","worker_backend":"","participants":["","..."],"pass_count":} ``` #### Step 8: Write `metadata.md` Write to `$SESSION_DIR/metadata.md` containing: - Command name, start time, configured pass count - Worker backend, participant artifact IDs, and executor mapping - Resolved models only for `model-clis`, timeout setting when applicable - Git branch (`git branch --show-current`), commit ref (`git rev-parse --short HEAD`) - Artifact source (file path or "inline text"), first 120 chars of artifact Store `$SESSION_DIR` for use in all subsequent phases. #### Step 8b: Repo Guard — Capture Fingerprint Capture the repository state before any model runs. See `docs/repo-guard-protocol.md` Layer 2 for the full protocol. ```bash REPO_TOPLEVEL="$(git rev-parse --show-toplevel)" ``` ```bash REPO_HEAD="$(git -C "$REPO_TOPLEVEL" rev-parse HEAD)" ``` ```bash REPO_FINGERPRINT="$(git -C "$REPO_TOPLEVEL" status --porcelain)" ``` Write `$SESSION_DIR/repo-fingerprint.txt` containing the HEAD ref and status output. Store `$REPO_TOPLEVEL` for use in all subsequent phases. #### Step 9: Write Context Packet Write the Context Packet built in Phase 1b to `$SESSION_DIR/context-packet.md`. #### Step 10: Write Original Artifact Write the original artifact text to `$SESSION_DIR/original.md`. This preserves the unmodified input for reference throughout the refinement process. --- ## Phase 3: Pass 1 — Independent Critique and Improvement **Goal**: Send the original artifact to all available models for independent critique and improvement. ### Prompt Preparation Each model receives a distinct evaluation lens to decorrelate critiques and reduce shared blind spots. The same context packet is included for all models, but a different role preamble is prepended to each prompt. | Slot | Role | Bias | Preamble | |------|------|------|----------| | Claude | **Maintainer** | Conservative, convention-enforcing, minimal-change | "You are the Maintainer. Prioritize correctness, convention adherence, and minimal scope. Challenge any change that isn't strictly necessary. Enforce all project conventions from CLAUDE.md/AGENTS.md." | | Antigravity | **Skeptic** | Challenge assumptions, find edge cases, question necessity | "You are the Skeptic. Challenge every assumption. Find edge cases, failure modes, and unstated requirements. Question whether the proposed approach is even the right one. Prioritize what could go wrong." | | GPT | **Builder** | Pragmatic, shippable, favor simplicity over abstraction | "You are the Builder. Prioritize practical, shippable solutions. Favor simplicity over abstraction. Focus on what gets the job done with the least complexity. Call out over-engineering." | Role preambles are prepended before the task-specific prompt and context packet. The role does not change the task — it changes the lens through which the model approaches critique and improvement. ### Critique and Improvement Prompt Each model receives the following prompt structure (with its role preamble prepended): > You are reviewing an artifact for iterative refinement. Produce THREE clearly separated sections: > > ## Critique > Analyze the artifact. What is strong? What is weak? What is missing? Be specific — quote passages, cite concrete issues. > > ## Improved Version > Produce your complete improved version of the artifact. This must be a full replacement, not a diff or partial edit. > > ## Rationale > Explain why you made each change. Connect each modification back to a specific finding in your critique. > > --- > > **Artifact to refine:** > > Include the context packet from Phase 1b. Write the prompt content to `$SESSION_DIR/pass-0001/prompt.md` using the Write tool. ### Claude Critique (Task agent) Launch a Task agent with `subagent_type: "general-purpose"` to critique and improve the artifact: **Prompt for the Claude agent**: > Critique and improve the following artifact. Read any relevant files to give a thorough, accurate assessment. Read CLAUDE.md/AGENTS.md for project conventions. > > Produce THREE clearly separated sections: Critique (what's strong, weak, missing), Improved Version (complete replacement), and Rationale (why each change was made). > > Artifact: > > CRITICAL: Do NOT write, edit, create, or delete any files in the repository. Do NOT use Write, Edit, or Bash commands that modify repository files. All session artifacts are written to `$SESSION_DIR`, which is outside the repository. This is a READ-ONLY research task. ### Antigravity Critique (sub-agent) Launch a Task agent (`subagent_type: "general-purpose"`, `mode: "default"`) to execute the Antigravity (agy) model. Include in the agent prompt: the resolved backend command and timeout from Phase 2, the `$SESSION_DIR` path, the pass number, the `$REPO_TOPLEVEL` path and `$REPO_FINGERPRINT` value for repo guard verification, and the critique instructions. Additional instructions for the Antigravity agent prompt: CRITICAL: Do NOT write, edit, create, or delete any files. Do NOT use any file-writing or file-modification tools. This is a READ-ONLY research task. All output must go to stdout. Any file modifications will be automatically detected and reverted. The agent must: 1. Read the prompt from `$SESSION_DIR/pass-0001/prompt.md` 2. Run the resolved Antigravity command with output redirection. **Repo Guard**: `agy` has no native read-only mode (its print mode reads *and* writes), so isolate it in a disposable git worktree checked out at `HEAD` — agy reads the snapshot while any stray write lands in the throwaway worktree, never the main repo (see `docs/repo-guard-protocol.md` Layer 1). The `gemini` and `agent` fallbacks keep their own native read-only modes. **Primary (`agy` CLI, disposable worktree)**: ```bash (AGY_RO_WT="${REPO_TOPLEVEL}-weave-agy-ro"; git -C "$REPO_TOPLEVEL" worktree remove --force "$AGY_RO_WT" 2>/dev/null; git -C "$REPO_TOPLEVEL" worktree add -q --detach "$AGY_RO_WT" HEAD && (cd "$AGY_RO_WT" && agy --model "Gemini 3.1 Pro (High)" --add-dir "$AGY_RO_WT" --dangerously-skip-permissions -p "$(cat "$SESSION_DIR/pass-0001/prompt.md")" "$SESSION_DIR/pass-0001/outputs/agy.md" 2>"$SESSION_DIR/pass-0001/stderr/agy.txt"); rc=$?; git -C "$REPO_TOPLEVEL" worktree remove --force "$AGY_RO_WT" 2>/dev/null; exit "$rc") ``` **Fallback (`gemini` CLI)**: ```bash (cd "$SESSION_DIR" && gemini -m gemini-3-pro-preview --approval-mode plan --include-directories "$REPO_TOPLEVEL" --skip-trust -p "$(cat "$SESSION_DIR/pass-0001/prompt.md")" >"$SESSION_DIR/pass-0001/outputs/agy.md" 2>"$SESSION_DIR/pass-0001/stderr/agy.txt") ``` **Fallback (`agent` CLI)**: ```bash (cd "$SESSION_DIR" && agent -p --mode plan --trust --workspace "$REPO_TOPLEVEL" --model gemini-3.1-pro "$(cat "$SESSION_DIR/pass-0001/prompt.md")" >"$SESSION_DIR/pass-0001/outputs/agy.md" 2>>"$SESSION_DIR/pass-0001/stderr/agy.txt") ``` 3. **Repo Guard**: After the CLI returns, verify the repository is unchanged (see `docs/repo-guard-protocol.md` Layer 3): ```bash CURRENT_STATUS="$(git -C "$REPO_TOPLEVEL" status --porcelain)" ``` ```bash if [ "$CURRENT_STATUS" != "$REPO_FINGERPRINT" ]; then echo "REPO GUARD VIOLATION: agy modified repository files" >&2 git -C "$REPO_TOPLEVEL" checkout -- . 2>/dev/null git -C "$REPO_TOPLEVEL" clean -fd 2>/dev/null printf '{"event":"repo_guard_violation","timestamp":"%s","model":"agy","reverted":true}\n' "$(date -u '+%Y-%m-%dT%H:%M:%SZ')" >>"$SESSION_DIR/guard-events.jsonl" fi ``` 4. On failure: classify (timeout → retry with 1.5x timeout; rate-limit → retry after 10s; credit-exhausted → skip retry, escalate to the next backend immediately; crash → not retryable; empty → retry once), retry max once with same backend, then fall back down the chain (agy → gemini → agent) if a native CLI was used; if all are credit-exhausted or unavailable, use the lesser model (`Gemini 3.5 Flash (High)` via agy for Antigravity, then `gemini-3-flash-preview`; gpt-5.4-mini via agent for GPT) 5. Return: exit code, elapsed time, retry count, output file path ### GPT Critique (sub-agent) Launch a Task agent (`subagent_type: "general-purpose"`, `mode: "default"`) to execute the GPT model. Include in the agent prompt: the resolved backend command and timeout from Phase 2, the `$SESSION_DIR` path, the pass number, the `$REPO_TOPLEVEL` path and `$REPO_FINGERPRINT` value for repo guard verification, and the critique instructions. Additional instructions for the GPT agent prompt: CRITICAL: Do NOT write, edit, create, or delete any files. Do NOT use any file-writing or file-modification tools. This is a READ-ONLY research task. All output must go to stdout. Any file modifications will be automatically detected and reverted. The agent must: 1. Read the prompt from `$SESSION_DIR/pass-0001/prompt.md` 2. Run the resolved GPT command with output redirection: **Repo Guard**: invoke the CLI in its native read-only sandbox — it reads the repo but cannot write it (see `docs/repo-guard-protocol.md` Layer 1) **Native (`codex` CLI)**: ```bash (cd "$SESSION_DIR" && codex exec -s read-only -C "$REPO_TOPLEVEL" --skip-git-repo-check "$SESSION_DIR/pass-0001/outputs/gpt.md" 2>"$SESSION_DIR/pass-0001/stderr/gpt.txt") ``` **Fallback (`agent` CLI)**: ```bash (cd "$SESSION_DIR" && agent -p --mode plan --trust --workspace "$REPO_TOPLEVEL" --model gpt-5.4-high "$(cat "$SESSION_DIR/pass-0001/prompt.md")" >"$SESSION_DIR/pass-0001/outputs/gpt.md" 2>>"$SESSION_DIR/pass-0001/stderr/gpt.txt") ``` 3. **Repo Guard**: After the CLI returns, verify the repository is unchanged (see `docs/repo-guard-protocol.md` Layer 3): ```bash CURRENT_STATUS="$(git -C "$REPO_TOPLEVEL" status --porcelain)" ``` ```bash if [ "$CURRENT_STATUS" != "$REPO_FINGERPRINT" ]; then echo "REPO GUARD VIOLATION: gpt modified repository files" >&2 git -C "$REPO_TOPLEVEL" checkout -- . 2>/dev/null git -C "$REPO_TOPLEVEL" clean -fd 2>/dev/null printf '{"event":"repo_guard_violation","timestamp":"%s","model":"gpt","reverted":true}\n' "$(date -u '+%Y-%m-%dT%H:%M:%SZ')" >>"$SESSION_DIR/guard-events.jsonl" fi ``` 4. On failure: classify (timeout → retry with 1.5x timeout; rate-limit → retry after 10s; credit-exhausted → skip retry, escalate to agent CLI immediately; crash → not retryable; empty → retry once), retry max once with same backend, then fall back to agent CLI if native was used; if agent is also credit-exhausted or unavailable, use lesser model (`Gemini 3.5 Flash (High)` via agy for Antigravity; gpt-5.4-mini via agent for GPT) 5. Return: exit code, elapsed time, retry count, output file path ### Artifact Capture After each model completes, persist its output to the session directory: - **Claude**: Write the Task agent's response to `$SESSION_DIR/pass-0001/outputs/claude.md` - **Antigravity**: Written by the Antigravity sub-agent to `$SESSION_DIR/pass-0001/outputs/agy.md` - **GPT**: Written by the GPT sub-agent to `$SESSION_DIR/pass-0001/outputs/gpt.md` ### Execution Strategy - Launch all model agents in the same turn to execute simultaneously. If parallel dispatch is unavailable, launch sequentially — the judge phase handles partial results. - Each sub-agent handles its own retry and fallback protocol internally (see steps 3-4 in each agent's instructions above). - After all agents return, verify output files exist in `$SESSION_DIR/pass-0001/outputs/`. - If a sub-agent reports failure after exhausting retries, mark that model as unavailable for this pass and include failure details in the report. - Never block the entire workflow on a single model failure. --- ## Phase 4: Judge-Weave-Distribute Cycle **Goal**: After each pass's model outputs are collected, judge the results, weave the best version, and distribute for the next pass. This phase runs after every pass (including pass 1). For the final pass, skip the Distribute step. ### Step 1: Judge **Determine this pass's judge.** If `judge_mode` is `"host"`, Claude judges every pass. If `"round-robin"`, build a rotation array from available models starting with Claude: `[claude, agy, gpt]` (skipping any model not detected in Phase 2 Step 3). The judge for pass N is `rotation[(N - 1) % len(rotation)]`. **Self-judging note**: In round-robin mode, the judge model is also a participant whose output is being judged. The host agent should cross-check the external judge's winner selection against its own reading of the outputs during the weave step. If the external judge selected its own output as winner and the host's assessment disagrees, the host may override the winner selection for weaving purposes. Record any override in `judge.md` with a note: `**Override**: Host overrode external judge's self-selection of — .` #### Host Judge Protocol (Claude judges) When the judge is Claude (host), the host agent reads ALL model outputs from the current pass and produces a structured assessment. **Read all outputs**: Read each file from `$SESSION_DIR/pass-NNNN/outputs/.md`. Extract the three sections (Critique, Improved Version, Rationale) from each. **Score each improved version** on four dimensions, 0-10: | Dimension | Description | |-----------|-------------| | Quality | Writing quality, clarity, precision, technical accuracy | | Originality | Novel improvements, creative solutions, fresh perspectives | | Completeness | Covers all aspects, nothing important missing | | Coherence | Internal consistency, logical flow, well-structured | Compute the total score for each model (sum of four dimensions, max 40). **Pick the winner**: The model with the highest total score. In case of tie, prefer the model whose Improved Version addresses the most critique points. **Write the judge's assessment** to `$SESSION_DIR/pass-NNNN/judge.md` in this format: ```markdown # Judge's Assessment — Pass NNNN **Judged by**: Claude (host) ## Scores | Dimension | | | ... | |-----------|-----------|-----------|-----| | Quality (0-10) | X | X | ... | | Originality (0-10) | X | X | ... | | Completeness (0-10) | X | X | ... | | Coherence (0-10) | X | X | ... | | **Total** | XX | XX | ... | Adjust table columns to include only participating models for this pass (e.g., `| Claude | Antigravity |` when only two models are available). ## Winner **** with a score of XX/40. ## Rationale ## Runner-Up Analysis ### - ``` #### External Judge Protocol (Antigravity or GPT judges) When the judge is an external model, dispatch a judging prompt to that model via CLI. **Step A: Construct judge prompt.** Build a prompt containing: 1. The scoring rubric (4 dimensions with descriptions, same table as above) 2. The expected output format (markdown scores table, Winner section, Rationale section, Runner-Up Analysis section with specific strengths per non-winning model) 3. ALL model outputs from this pass, included **inline** — external models cannot read session files 4. Instruction: "You are judging these outputs. Score each on four dimensions (0-10), pick the winner, explain your rationale, and identify specific strengths in each non-winning version that the winner lacks." The judge prompt structure: > You are the judge for this refinement pass. You have received improved versions of an artifact from multiple AI models. Your job is to evaluate them fairly and pick the best one. > > **Scoring rubric** — score each version 0-10 on these dimensions: > > | Dimension | Description | > |-----------|-------------| > | Quality | Writing quality, clarity, precision, technical accuracy | > | Originality | Novel improvements, creative solutions, fresh perspectives | > | Completeness | Covers all aspects, nothing important missing | > | Coherence | Internal consistency, logical flow, well-structured | > > **Output format** — produce your assessment in this exact format: > > The table columns and model output sections below are illustrative. Adjust them to include only the models that produced outputs for this pass. > > ``` > ## Scores > > | Dimension | | | ... | > |-----------|-----------|-----------|-----| > | Quality (0-10) | X | X | ... | > | Originality (0-10) | X | X | ... | > | Completeness (0-10) | X | X | ... | > | Coherence (0-10) | X | X | ... | > | **Total** | XX | XX | ... | > > ## Winner > > **** with a score of XX/40. > > ## Rationale > > > > ## Runner-Up Analysis > > ### > - > > ### > - > ``` > > --- > > **Model outputs to judge:** > > For each participating model, include a section: > > ### 's Output > **Step B: Write and dispatch judge prompt.** Write the prompt to `$SESSION_DIR/pass-NNNN/judge-prompt.md`. Dispatch to the judge model using the same CLI mechanism as participation: **Antigravity as judge** — primary (`agy` CLI, disposable worktree). `agy` has no native read-only mode, so isolate it in a disposable git worktree checked out at `HEAD`, in a SEPARATE worktree from any model-pass run so the two never collide (see `docs/repo-guard-protocol.md` Layer 1): ```bash (AGY_RO_WT="${REPO_TOPLEVEL}-weave-agy-ro-judge"; git -C "$REPO_TOPLEVEL" worktree remove --force "$AGY_RO_WT" 2>/dev/null; git -C "$REPO_TOPLEVEL" worktree add -q --detach "$AGY_RO_WT" HEAD && (cd "$AGY_RO_WT" && agy --model "Gemini 3.1 Pro (High)" --add-dir "$AGY_RO_WT" --dangerously-skip-permissions -p "$(cat "$SESSION_DIR/pass-NNNN/judge-prompt.md")" "$SESSION_DIR/pass-NNNN/judge-raw.md" 2>"$SESSION_DIR/pass-NNNN/stderr/judge-agy.txt"); rc=$?; git -C "$REPO_TOPLEVEL" worktree remove --force "$AGY_RO_WT" 2>/dev/null; exit "$rc") ``` **Antigravity as judge** — fallback (`gemini` CLI): ```bash (cd "$SESSION_DIR" && gemini -m gemini-3-pro-preview --approval-mode plan --include-directories "$REPO_TOPLEVEL" --skip-trust -p "$(cat "$SESSION_DIR/pass-NNNN/judge-prompt.md")" >"$SESSION_DIR/pass-NNNN/judge-raw.md" 2>"$SESSION_DIR/pass-NNNN/stderr/judge-agy.txt") ``` **Antigravity as judge** — fallback (`agent` CLI): ```bash (cd "$SESSION_DIR" && agent -p --mode plan --trust --workspace "$REPO_TOPLEVEL" --model gemini-3.1-pro "$(cat "$SESSION_DIR/pass-NNNN/judge-prompt.md")" >"$SESSION_DIR/pass-NNNN/judge-raw.md" 2>>"$SESSION_DIR/pass-NNNN/stderr/judge-agy.txt") ``` **GPT as judge** (native CLI): ```bash (cd "$SESSION_DIR" && codex exec -s read-only -C "$REPO_TOPLEVEL" --skip-git-repo-check "$SESSION_DIR/pass-NNNN/judge-raw.md" 2>"$SESSION_DIR/pass-NNNN/stderr/judge-gpt.txt") ``` Use the same fallback and retry protocol as participation dispatch (see Phase 3). If the native CLI fails, fall back to `agent --model `. **Repo Guard**: After each external judge CLI returns, verify the repository is unchanged using the same post-CLI verification as Phase 3 (see `docs/repo-guard-protocol.md` Layer 3). Log any violation with the judge model name. **Step C: Parse external judge output.** Read `$SESSION_DIR/pass-NNNN/judge-raw.md` and extract: 1. **Scores table**: Find the markdown table with dimension scores. Validate each score is an integer 0-10 and totals match the sum of dimensions. 2. **Winner**: Find the `## Winner` section. Validate the named model is a participating model. 3. **Rationale**: Extract the `## Rationale` section content. 4. **Runner-Up Analysis**: Extract the `## Runner-Up Analysis` section. Parse per-model strengths. **Step D: Validate and fallback.** If parsing fails — no valid scores table, no identifiable winner, winner is not a participating model, or the CLI command failed entirely — **fall back to the Host Judge Protocol** for this pass. Record the fallback. A missing or unparseable Runner-Up Analysis does NOT trigger full fallback; the host populates it in Step 2. **Step E: Write `judge.md`.** Write the parsed assessment to `$SESSION_DIR/pass-NNNN/judge.md` in the same format as the Host Judge Protocol, with the header: - Normal: `**Judged by**: (round-robin pass N)` - Fallback: `**Judged by**: Claude (fallback from — )` When the external judge's Runner-Up Analysis is thin or missing specific strengths, the host agent supplements with its own observations during the weave step. ### Step 2: Analyze Runners-Up When an external judge provided Runner-Up Analysis in `judge.md`, use it as the starting point and supplement if thin or missing specific strengths. When the host judged, produce the analysis from scratch. For each non-winning model's improved version, identify **specific strengths** it has that the winner lacks. Be concrete: - Quote specific passages or phrasings that are better - Identify techniques, approaches, or structural choices worth incorporating - Note any critique points that this model addressed but the winner did not Record these strengths — they feed directly into the weave step. ### Step 3: Weave Produce a woven version that combines the best of all models: 1. **Start from the winner's improved version** as the base 2. **Incorporate identified strengths** from runners-up — integrate specific passages, techniques, or structural improvements 3. **Address weaknesses** noted in the winner's own critique section — if the winner's critique identified issues in its own improved version, fix them 4. **Preserve coherence** — the woven result must read as a unified artifact, not a patchwork of different styles Write the woven version to `$SESSION_DIR/pass-NNNN/woven.md`. ### Step 4: Distribute (skip for final pass) If this is NOT the final pass, distribute the woven version back to ALL models for another round of critique and improvement. **Create the next pass directory**: ```bash mkdir -p -m 700 "$SESSION_DIR/pass-$(printf '%04d' $NEXT_PASS)/outputs" "$SESSION_DIR/pass-$(printf '%04d' $NEXT_PASS)/stderr" ``` **Construct the distribution prompt** for the next pass. Each model receives: > You previously critiqued and improved an artifact. The judge has evaluated all versions and produced a woven result incorporating the best elements from each model's contribution. > > **Woven version** (the current best): > > > > **Judge's rationale**: > > **Strengths incorporated from other models**: > - > - > > **Weaknesses addressed**: > - > > **Residual focus** — unresolved items from this pass: > - > > --- > > Critique this woven version further and produce an improved version. Use the same three-section format: > > ## Critique > Address ONLY the residual-focus items plus any new defect the weave itself introduced. Do not re-litigate settled points. Be specific. > > ## Improved Version > Produce your complete improved version. This mus …(truncated)