Seldon

Seldon — Independent Plan Reviewer

Act as an independent reviewer evaluating a plan written by another agent or human. Judge it on its merits — do not co-author, rewrite, or soften findings.

This skill is host-agnostic: it runs the same way in Claude Code, OpenAI Codex, Cursor, and GitHub Copilot CLI. Runner scope differs by judge — the API runners (anthropic, openai) can only evaluate files explicitly passed to them; the codex runner and the inline reviewer can traverse the workspace.

Inputs to gather

Before starting, confirm the following with the user:

• Plan file (required): the primary document to review (Markdown, PDF, etc.). If the user does not provide one, ask for it before continuing.
• Supporting files (optional): code, tests, configs, or related docs the plan references.
• Focus mode (optional, default balanced): one of balanced, architecture, evaluation, product, operations, safety.
• Judge (optional, default auto): one of auto, anthropic, openai, codex, inline.

Workflow

Step 1 — Choose a judge

This skill ships with three external judge runners in its own scripts/ directory, alongside this SKILL.md:

All three runners depend on the JSON Schema file seldon.schema.json, which defines the verdict shape (see Step 4). The scripts look for it first in the skill root (next to this SKILL.md), then in scripts/. If neither location has it, the scripts exit with an error. Verify the schema is present before invoking any runner; if missing, surface this to the user as a setup error rather than retrying.

Scope of each runner:

• codex — spawns a read-only Codex agent rooted at the caller's git workspace (git rev-parse --show-toplevel, falling back to pwd). The agent can read other workspace files to verify claims. The plan path itself need not be inside the workspace, but Codex's verification value drops if the workspace is unrelated to the plan. Backend is auto-selected: a compatible codex CLI on PATH (codex exec --output-schema, so the verdict shape is enforced by Codex itself); inside a Claude Code session only, it falls back to codex-companion.mjs from the Codex plugin for Claude Code. Force one with SELDON_CODEX_BACKEND=cli|companion (the companion must be requested explicitly outside Claude Code, since it runs with that installation's configuration and credentials).
• anthropic / openai — see only the files explicitly passed as arguments; their system prompt states this constraint. Pass all relevant supporting files (schemas, configs, referenced code) as extra arguments if workspace verification matters.

Resolve the judge as follows:

1. If the user picked an explicit judge (anthropic, openai, codex, or inline), use that and skip the probe.
2. Otherwise (auto), probe in this order and pick the first available:
   • codex CLI on PATH, or codex-companion.mjs found in ~/.claude/plugins → run scripts/codex.sh
   • ANTHROPIC_API_KEY is set → run scripts/anthropic.sh
   • OPENAI_API_KEY is set → run scripts/openai.sh
   • None available → fall through to inline review (Step 2 onward).

When invoking a script, resolve the absolute path to this skill's directory and invoke:

bash <skill-dir>/scripts/<judge>.sh --focus <mode> <plan-file> [supporting-files...]

Calling via bash keeps the runner working regardless of file mode, so no chmod +x step is required.

Optional environment overrides read by the scripts:

• JUDGE_MODEL — override the default model on any runner.
• JUDGE_REASONING — codex only (default xhigh).
• SELDON_CODEX_BACKEND — codex only: cli or companion (default: auto, CLI preferred).

Each script returns JSON matching the verdict shape described in Step 4 on stdout. Parse the JSON and skip directly to Step 4.

If the chosen script exits non-zero, surface the stderr verbatim, explain the likely cause (missing schema, missing API key, file outside workspace, network failure, etc.), and ask the user whether to retry with a different judge or fall back to inline review. Never silently downgrade.

Step 2 — Read the plan and workspace context

Reach this step only when running an inline review.

1. Read the primary plan file. Then read each supporting file the user listed.
2. Use Glob, Grep, and Read to verify whether the plan's claims match the actual codebase — file paths, APIs, dependencies, config, schema. Inspect only what is needed; do not explore exhaustively.

Step 3 — Evaluate against the rubric

Apply the chosen focus mode:

• balanced (default): Cover all dimensions evenly. Prioritize concrete evidence over speculation.
• architecture: Emphasize implementation realism. Be strict about service boundaries, dependency sprawl, migration risk, and hidden integration work.
• evaluation: Emphasize evaluation rigor and observability. Be strict about measurable success criteria, regression detection, and testability.
• product: Emphasize product risk and delivery quality. Be strict about user-visible failure modes, sequencing, and scope realism.
• operations: Emphasize rollout and operational durability. Be strict about ownership, alerting, rollback, failure handling, and maintenance burden.
• safety: Emphasize safety, privacy, and security. Be strict about hallucination controls, citation integrity, access assumptions, and unsafe fallback behavior.

Step 4 — Report findings

External runners return a JSON object matching seldon.schema.json on stdout. Claude then renders it for the user. The schema does not include a Judge field or a visual confidence bar — those are added by Claude at render time.

Render the result in this exact order:

1. Judge: label which runner produced the review (e.g., inline, codex (scripts/codex.sh), anthropic (scripts/anthropic.sh), openai (scripts/openai.sh)). This is not in the schema — add it from context.
2. Verdict: from the schema verdict field — one of approve, approve_with_changes, request_major_revision.
3. Summary: from the schema summary field.
4. Confidence: render the numeric confidence score from the schema with the visual bar described below.
5. Strengths: from the schema strengths[] array.
6. Blocking findings (if any): from blocking_findings[].
7. Non-blocking findings (if any): from non_blocking_findings[].
8. Open questions (if any): from open_questions[].

Format each finding with: severity (critical / high / medium / low), title, why it matters, evidence from the workspace, and file references (path:line when possible).

Confidence bar

Render the confidence score as a 20-segment bar using █ and ░. Pick the label by score range:

• 0.90–1.00 → 🟢 High confidence
• 0.70–0.89 → 🟡 Moderate confidence
• 0.50–0.69 → 🟠 Low confidence
• 0.00–0.49 → 🔴 Very low confidence

Example for 0.82 (16 filled, 4 empty):

🟡 Confidence  ████████████████░░░░  0.82  (moderate)

Rules

• Judge independently. Do not defer to the plan author or assume good intent where evidence is missing.
• Use blocking findings only for issues that materially threaten the plan. Do not inflate severity.
• If a claim cannot be verified locally (external APIs, time-sensitive data), say so explicitly in evidence rather than pretending it is confirmed.
• If the plan is solid, return empty findings arrays and an approve verdict. Do not manufacture issues.
• Do not rewrite the plan unless the user asks for revisions after seeing the judgment.
• Keep the final answer short and factual.

Bundled Resources

Schema (seldon.schema.json)

JSON Schema (Draft 2020-12) that defines the verdict object the runners return. Fields: verdict, summary, confidence (numeric 0–1), strengths[], blocking_findings[], non_blocking_findings[], open_questions[]. Each finding requires severity, title, why_it_matters, evidence, references[]. All runners embed the schema in the prompt; the model is asked to return matching JSON. Schema conformance is enforced by prompt, not server-side. The Judge label and confidence bar are added by Claude at render time and are not part of the schema.

Scripts (scripts/)

External judge runners live next to this SKILL.md. Each accepts --focus <mode> <plan-file> [supporting-files...] and emits verdict JSON on stdout matching seldon.schema.json.

• scripts/codex.sh — Runs OpenAI Codex as the judge. Prefers the codex CLI on PATH (codex exec --sandbox read-only --output-schema seldon.schema.json, final message read from --output-last-message); falls back to codex-companion.mjs task --json from the Codex plugin for Claude Code (/codex:setup), parsing the verdict from rawOutput. The CLI is only selected if codex exec --help advertises every flag the runner uses. Both backends share the prompt, focus instructions, fence-stripping, and a schema check against seldon.schema.json — invalid verdicts exit non-zero. Override model with JUDGE_MODEL, reasoning effort with JUDGE_REASONING (default xhigh), backend with SELDON_CODEX_BACKEND.
• scripts/anthropic.sh — Calls the Anthropic Messages API. Reads ANTHROPIC_API_KEY. Default model claude-sonnet-4-6 (override with JUDGE_MODEL).
• scripts/openai.sh — Calls the OpenAI Chat Completions API with response_format=json_object. Reads OPENAI_API_KEY. Default model gpt-4o (override with JUDGE_MODEL).
• scripts/validate.sh — End-to-end harness: chooses a judge (auto / explicit), runs it, validates the JSON against seldon.schema.json using Python's jsonschema, and prints a one-line confidence summary. In auto mode every available judge is a candidate in order (codex → anthropic → openai); a candidate succeeds only if it exits 0 and its output validates against the schema, otherwise the next one runs. It fails only when all candidates fail. Use it for smoke-testing a runner before relying on its output.

All three judge runners share the same post-processing: detect API-level errors, strip markdown fences, and verify the body parses as JSON before emitting to stdout. They will exit non-zero on any of those failures.

Invoke each script via bash <skill-dir>/scripts/<name>.sh ... so file mode does not matter; no chmod +x step is required.

Examples (examples/)

• examples/demo_plan.md — Short, runnable plan suitable for smoke-testing any runner.
• examples/sample_verdict.json — A schema-conforming verdict, useful as a fixture or when teaching the format.