# Idea Spark

> Generate ONE reviewer-defensible, implementable research idea with a concrete method and falsification plan from a stated research direction. Use when the user asks for a research idea, novelty analysis, bottleneck diagnosis, or paper-shape suggestion. Skip code review, debugging, and unconstrained brainstorming without research context.

- Skill: `microsoft/idea-spark` (Agent Skill, multi-file: 105 files)
- Install (CLI): `npx skillmds@latest add microsoft/idea-spark`
- Raw SKILL.md: https://api.skillmd.com/api/skills/microsoft/idea-spark/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Research & Search
- Author: Microsoft (https://skillmd.com/u/microsoft)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/microsoft/idea-spark

---


# Idea Spark Skill

Convert an under-specified research direction into ONE reviewer-defensible Oral-level research proposal — grounded in 1947 ICLR/ICML/NeurIPS papers (2021-2025) — via a 5-phase workflow: retrieve recent literature, diagnose the bottleneck, select + generate a candidate using corpus-derived ideation pattern cards, run it through a quality gauntlet, expand into an idea card.

This file is the operational runbook. Design rationale (the 7 design principles, why each contract is shaped this way, removed-check history) lives in [references/design-notes.md](references/design-notes.md) — read it only when modifying or evaluating the skill, never needed to run it. When MODIFYING the skill, also replay the cross-shape regression set in [references/regression-directions.md](references/regression-directions.md) (deterministic subset: `python3 "$SKILL_DIR/scripts/regression_check.py" <run_dir>`; routing/merger branch fixtures — including every guard/retry branch that real runs rarely exercise: `python3 "$SKILL_DIR/scripts/selftest_routing.py"`; unit fixtures for the helpers every phase sits on — the multi-query round-robin and the tolerant LLM-JSON loader: `python3 "$SKILL_DIR/scripts/selftest_units.py"`). First-time installation lives in [references/setup.md](references/setup.md).

## When to use

- "Give me a research idea in {area} I could pursue." / "What's the most impactful next step in this direction?"
- "Help me sharpen this vague direction into an Oral-level proposal."
- "What's the bottleneck of this problem?" / "Run a novelty audit on this idea."

## When NOT to use

- Code review, debugging, refactoring. Summarizing one paper. Cross-decade survey writing.
- Free-association brainstorming with no research context. Engineering integration tasks ("ship this feature in our system").
- Pure benchmark / dataset construction work — the 15-pattern vocabulary handles benchmark *audit* (controlled_diagnostic_design) but not benchmark *construction*.

## Setup (first use only)

Follow [references/setup.md](references/setup.md). Quick version — set two shell variables, install deps, verify:

```bash
SKILL_DIR=<path to this folder>                  # e.g. ~/.claude/skills/idea-spark (Claude Code), ~/.codex/skills/idea_spark (Codex CLI), or any clone location
RUN_DIR="$PWD/ideaspark_run/<topic-slug>"   # convention below; any absolute dir works. No mkdir needed — every phase mkdir -p's its own --out, and `next` treats a missing dir as a fresh run
python3 -m pip install feedparser openreview-py beautifulsoup4 pymupdf
python3 "$SKILL_DIR/scripts/run.py" check_connectors   # from the SAME shell you'll run phases from
```

Credentials go in `.env` (OpenReview user/pass + Semantic Scholar key — see setup.md); the orchestrator auto-loads it. Optional: `xelatex`/`tectonic` for PDF cards.

---

## How to run: the `next` loop

The canonical way to drive a run is the **run-state navigator**:

```bash
python3 "$SKILL_DIR/scripts/run.py" next --dir "$RUN_DIR" --query "<user's research question>"
```

**Run-dir convention (one run = one directory, named by the host BEFORE the first command):** `$PWD/ideaspark_run/<topic-slug>` — a short kebab-case slug distilled from the user's direction (e.g. `ideaspark_run/diffusion-watermark`); if the slug is taken, append `_2`, not a timestamp. NEVER reuse a directory that already contains a `phase0/` — every phase writes into `$RUN_DIR` and would clobber the prior run. The skill itself never names the dir (any absolute path works); this convention exists so runs from different harnesses land in one predictable place instead of each agent improvising.

`next` inspects the artifacts already on disk and prints EXACTLY one next step — either a Bash command to run verbatim, or an LLM sub-agent spec (system-prompt path + input file paths + output path + the routing signal to report back). It is read-only and idempotent (safe to re-run anytime, including to resume an interrupted run) — including on a run dir that does not exist yet, which it reports as a fresh run and answers with the Phase 0 step rather than an error. You never need to create the directory before calling it. The host loop is:

1. Run `next`.
2. Do what it says (`bash` → run the command; `llm_subagent` → execute in an ISOLATED context per the Context discipline rules below).
3. Run `next` again. Repeat until it reports a terminal state (`DONE`, `do_not_generate`, or `phase_3_failed`).

**Consume every emit WHOLE — never grep/filter/truncate the block.** INPUT lists span unlabeled continuation lines; a label-keyed grep (`grep -E "STEP|INPUT|..."`) silently drops them. This exact failure occurred in a live run: the navigator listed the coherence gate's blocking findings as an audit input, the host's grep dropped the line, and the audit issued a false `advance` without ever seeing the executed evidence. If the emit must be captured into a bounded tool result, `head -c 4000` the WHOLE block — never filter by line labels.

`next` encodes the full phase graph — the mandatory full-text gate, the citation gate, the abandon-retry branch, the falsification re-audit branch, and the correct Phase 4 flags per path — so you do not need to memorize the reference tables below; they exist for deviation and debugging.

If your host exposes a task/todo tool (e.g., TodoWrite), seed it with this checklist and tick phases as `next` moves past them:

```
- [ ] Phase 0: Literature grounding → lit_table.md, then Phase 0+ full-text fetch (MANDATORY — Phase 1 hard-gates on it)
- [ ] Phase 1: Bottleneck identification → phase1_output.json (routing: proceed | do_not_generate)
- [ ] Phase 2: Gap×pattern selection + candidate generation (ONE isolated context, TWO output files) → citation gate → coherence gate (dry-run trace, fresh context)
- [ ] Phase 3: Collision retrieval (signature@10mo + alias@48mo — launchable in parallel with 2.3) → audit (5 checks) → [revise → merge → re-audit if falsification rewritten] | [abandon → information-gain retry: regenerate while each failure yields NEW binding lessons, ≤3 candidate cycles; repeated subsumption lesson → 1 bottleneck re-diagnosis + 1 attempt; no new information or cap → phase_3_failed]
- [ ] Phase 4: skeleton → fill (technical) → assemble partial → derive (plain, fast-tier) → assemble final + method view → implementability audit → validate → render → return 3 cards inline
```

Three outcomes per run: the rendered idea-card markdown returned inline (LaTeX + per-phase JSON left under `$RUN_DIR`), a `do_not_generate.md` (Phase 1 OOD), or a `phase_3_failed.md` (audit abandons twice). **Never ask the user mid-flow** — missing intake fields are inferred; revision, falsification re-audit, and the single internal retry all run without user re-invocation.

### Invocation contract

**No `cd` is required.** `scripts/run.py` self-locates its skill root, so every orchestrator command can be invoked from ANY working directory by absolute script path: `python3 "$SKILL_DIR/scripts/run.py" <subcommand> --out "$RUN_DIR/<phase>/" ...`. The legacy form `cd "$SKILL_DIR" && python3 -m scripts.run <subcommand> ...` works identically. Do NOT use relative script or `--out` paths — CWD is not stable across host-LLM Bash invocations, and the orchestrator rejects a relative `--out` outright.

**Exit codes 10 and 11 are NOT errors — they are sentinel handshakes.** When the orchestrator can't call an LLM itself (no `NOVELTY_LLM_CLASSIFY_FAST_CMD`), it writes a sentinel JSON describing what the host LLM should do, then exits rc=10 (intent / pattern-summary) or rc=11 (signature_terms). Read the sentinel (`$RUN_DIR/<phase>/.<step>_pending`), read the file at its `rubric_file` field (absolute path), produce the expected output, re-invoke per its `re_invocation` field. Do not stop on these codes. (The default Phase 0 flow below avoids the rc=10 intent sentinel entirely by passing `--queries` up front.)

### Context discipline (read BEFORE running any LLM-driven phase)

A full run accumulates ~180-250k tokens of intermediate state. If the host LLM carries that in its own conversation context across phases, the Phase 1 / 2.2 / 4.fill calls routinely hit the backend request timeout (`[API Error · Request timed out · Retrying...]`) and the retry times out again. Apply ALL three rules on every run:

**Rule 1 — Run every LLM-driven phase in an ISOLATED context.** Phases 1 / 2 (2.1+2.2) / 2.3 / 3.2 / 3.3 / 4.fill / 4.1.5 each have file-path inputs and one JSON output; no phase needs the conversation that produced an earlier one. Use the FIRST isolation mechanism your harness supports:

- **(a) Subprocess LLM** — set `NOVELTY_LLM_REASONING_LARGE_CMD` / `NOVELTY_LLM_CLASSIFY_FAST_CMD` (see § Configuration); each phase runs as its own subprocess, fresh context by construction, on any harness.
- **(b) Sub-agent tool** (Claude Code `Agent` or equivalent) — spawn one per phase, passing ONLY the file paths the phase prompt lists — not conversation history, not file contents inline. The sub-agent reads from disk, `Write`s to disk, returns ≤ 250 words (output path + routing signal). Exception by design: Phase 2.1 and 2.2 run in ONE sub-agent writing both output files — both are generation-side; the adversarial separations (3.2 vs 3.3, 4.fill vs 4.1.5) must stay separate calls.
- **(c) Manual context reset** — run inline but clear/compact at the four points in Rule 3.

Whichever mechanism, the parent context stays ≤ ~30k tokens for the whole run because it never holds a phase's structured output.

**Rule 2 — `Write` every phase artifact directly to disk; never paraphrase it into chat.** Output convention: `$RUN_DIR/<phase>/<phase>_output.json`. Use your harness's file-write tool (Claude Code: `Write`) — no Bash heredocs (permission prompts + silent truncation), no `echo`, no pasting JSON into replies. Bound tool-result captures from large files to ≤ 4 KB (`head -c 4000` / `jq` / `sed`); never `Read` a >10 KB intermediate dump into the parent context — the dump gets cached into every subsequent turn (this exact anti-pattern caused prior timeout runs).

**Rule 3 — Compact between phases.** Natural compact points: after Phase 0+, after Phase 1, after Phase 2, after Phase 3.2. Every phase re-reads its disk inputs, so compacting loses nothing. With `/compact`, use it there; Rule 1 mechanisms (a)/(b) achieve the same on their own.

**Diagnostic for "Request timed out" mid-phase:** inspect your harness's session transcript/log (Claude Code: `~/.claude/projects/<project-slug>/<session-id>.jsonl`, look for `isApiErrorMessage: true`; other harnesses: their session-log equivalent); the prior tool call shows which prompt got too big. The fix is one of the three rules — usually Rule 1.

---

## Phase reference

`next` prints each of these steps at the right moment with concrete paths; the tables below are the full contract for deviation/debugging.

### Orchestrator entry points

| Phase | Entry point (`python3 "$SKILL_DIR/scripts/run.py" ...`, any CWD) |
|---|---|
| navigator | `next --dir "$RUN_DIR" [--query "..."]` |
| Phase 0 | `phase0 --query "<user text>" --queries "q1\|q2\|q3\|q4" [--named-papers "Title A\|Title B"] --out $RUN_DIR/phase0/` |
| user-ref registration (title-named anchor papers; BEFORE phase0_fulltext) | `add_user_ref --out $RUN_DIR/phase0/ --title "<full title>" [--raw-match "<user phrasing>"] [--id <arxiv/DOI/URL>]` |
| relevance partition apply (Phase 0.4; archives off_topic + stamps core/adjacent, BEFORE tagging) | `apply_partition --out $RUN_DIR/phase0/ --partition <relevance_partition.json>` |
| host-ref resolution (Phase 0.5 coverage check; verifies + merges host-nominated missing papers) | `add_host_refs --out $RUN_DIR/phase0/ --refs <noms.json>` |
| Phase 0+ full-text (**mandatory**, the moment lit_table.md lands) | `phase0_fulltext --out $RUN_DIR/phase0/` |
| Phase 1 anchor top-up (optional, when the #1 closest_adjacent fell outside the fulltext pool **or came back method-thin**) | `phase1_fulltext_topup --out $RUN_DIR/phase0/ --paper-id <anchor paper_id> [--min-method-chars 4000]` |
| Phase 2 prep (deterministic; `next` emits it with the Phase 2 step) | `phase2_prepare --dir $RUN_DIR` |
| lit_table shard assembly (deterministic; after parallel pattern tagging) | `lit_table_merge --out $RUN_DIR/phase0/ --shards <rows1.md> <rows2.md> ...` |
| Phase 3.1 collision | `phase3_collision --idea-json <canonical candidate> --out $RUN_DIR/phase3_collision/` |
| Phase 3.3 merger | `phase3_merge_revisions --phase2 <canonical candidate> --revisions <p3.3-patch> --critique <p3.2-report> --out $RUN_DIR/phase3_revise/` |
| Phase 2.3 merger (same tool; only when coherence verdict=patched) | `phase3_merge_revisions --phase2 <p2.2-output> --revisions <p2.3-output> --out $RUN_DIR/phase2_coherence/ --out-name refined_candidate.json` |
| Phase 4 skeleton | `phase4_skeleton --candidate <final_candidate-or-p2.2> --phase1 ... --phase2-select ... --phase3-critique ... [--phase3-revise ...] --phase0-dir $RUN_DIR/phase0/ [--collision ...] --out $RUN_DIR/phase4/` |
| Phase 4 assemble (repeat `--fill-map` to merge the derive_map on the final pass) | `phase4_assemble --skeleton $RUN_DIR/phase4/phase4_skeleton.json --fill-map $RUN_DIR/phase4/fill_map.json [--fill-map $RUN_DIR/phase4/derive_map.json] --out $RUN_DIR/phase4/` |
| Phase 4 method view (before 4.1.5; `next` bundles it with the final assemble) | `phase4_method_view --expansion $RUN_DIR/phase4/phase4_expansion.json --out $RUN_DIR/phase4/` |
| Phase 4 render | `phase4_render --expansion $RUN_DIR/phase4/phase4_expansion.json --out $RUN_DIR/phase4/` |
| Validators | `validate --phase2 ... [--phase3 ...] [--phase4 ...] [--phase4-impl ...]` |

The LLM-driven phases (1 / 2.1 / 2.2 / 2.3 / 3.2 / 3.3 / 4.fill / 4.1.5 / falsification re-audit) have no orchestrator subcommand (a `cat prompt | llm` wrapper would add fragility without determinism): read the prompt at `references/system-prompts/<phase>.txt`, gather the inputs listed at its top, `Write` the JSON described under `Output:` to `$RUN_DIR/<phase>/<phase>_output.json`. Run each under the Context discipline rules — Phase 4.fill is the largest output and the most timeout-prone; never in the parent context.

### Phase 0 — Literature grounding

Phase 0 and 3.1 require **real external retrieval** via the bundled connector scripts (`scripts/search_*.py`) — never WebSearch or ad-hoc fetch (downstream phases reject unstructured output). Gate sentinel: `.lit_grounding_mode` = `real` vs `connector_failure` (halt with diagnostic; `--allow-webfallback` exists as a flagged, lower-confidence escape).

**Default flow (skips one sentinel round-trip):** BEFORE invoking `phase0`, read `references/intent-recognition.md` (Map mode) yourself and produce **4** search queries (3-5 only with a stated reason) — including one ESCAPE-MECHANISM query phrased in solution vocabulary (recalls papers that already fixed the bottleneck and title themselves by their fix; problem-keyed queries miss exactly those), and apply that file's two query tests — VOCABULARY-OWNERSHIP and CONCRETE-OBJECT — to EVERY query, not only the escape one; its count rule gates a 5th query on quality per slot, not on the count. Phrase queries mechanism-first: generic "`<topic>` challenges/overview/landscape" phrasings are survey magnets that dilute corpus density — the orchestrator demotes survey-titled hits to the bottom of lit_results (kept, never dropped), but each connector's cap slots are still spent on them. Also apply the OOD short-circuit (intake-routing.md triggers #1 Too-broad / #2 No-anchor → route to do_not_generate instead of retrieving). Then invoke with BOTH flags:

```bash
python3 "$SKILL_DIR/scripts/run.py" phase0 --query "<user's research question>" --queries "q1|q2|q3|q4" --out $RUN_DIR/phase0/
```

The rc=10 sentinel path still exists as fallback when `--queries` is omitted. The orchestrator: asserts a sane clock; runs a list of retrieval JOBS (a connector may run more than one window) and merges them SS-priority. Default jobs: arxiv 0-6mo cap 40 · **ss_recent 0-6mo cap 30** (the freshness window is covered by TWO engines — arXiv's weak lexical API AND SS's stronger recent search — because gap freshness lives here and one retriever's ranking is a single point of failure) · openalex 6-24mo cap 30 published-only · semanticscholar 6-24mo cap 30 published-only · openreview 0-6mo cap 10 in-review · oa_recent 0-6mo **cap 0 (off by default)**. Caps are **wide on purpose** — narrower ones saturated, and the Phase 0.4 partition (below) drops the extra noise before it reaches the gap corpus (design-notes). ~140 pre-dedup, ~110-130 after dedup, ~50-75 after the partition drops off_topic. Window OVERLAP is intentional now (a second engine re-covers the freshness window; SS-first dedup collapses duplicates and enriches arXiv records with externalIds) — the old non-overlapping rule is retired. All caps are overridable per job via `IDEASPARK_POOL` (see § Configuration). Pass `--named-papers` whenever the query name-drops a paper or system without a link: the URL/ID regex only sees links, so a bare name is otherwise invisible to the U fetch tier and never becomes an anchor the candidate is differentiated against (add one later with `add_user_ref`). Each name is resolved against the connectors by CONTAINMENT — the paper's title must lead with the name, since a user writes the nickname and not the title — and a name two papers lead with is reported AMBIGUOUS and left unresolved rather than guessed, because a wrong U-tier entry is uncapped, always deep-read and becomes an anchor. Pass that one as a URL instead. It writes the user's question verbatim to `phase0/user_query.txt` — the 0.4 partition, the 0.5 coverage check, Phase 1 and Phase 2.1 are all pointed at that FILE rather than at a string the host retypes, so a paraphrase cannot silently propagate and a compacted run can still recover what was asked. It also extracts URL/ID user-refs from the query into `phase0/user_refs.json`; emits `.pattern_summary_pending` for the host.

**Phase 0.4 relevance partition (host precision gate; DEFAULT ON, disable with `IDEASPARK_RELEVANCE_PARTITION=off`):** because retrieval now over-fetches, the host — in ONE compact pass over every record's title+abstract — labels each `core | adjacent | off_topic`, writing `phase0/relevance_partition.json` `[{paper_id, relevance, reason}]`. Use YOUR OWN model (open-ended relevance judgment — do NOT downgrade); be CONSERVATIVE — when unsure between core and adjacent pick core, hard-label off_topic ONLY for clear cross-domain false positives (broad "memory-augmented" matches from wireless/recommendation/NLP), pure surveys, or unrelated fields (a wrong off_topic is an unrecoverable recall loss). Then `apply_partition --out phase0/ --partition <file>` (deterministic) stamps `relevance` on each record, **archives off_topic to `off_topic.md` and drops it from `lit_results.json`**, and touches `.partition_applied`, and writes **`query_yield.md`** — the Phase 0.4 labels joined back onto the `from_query` provenance the connectors stamp, i.e. each query's core/adjacent/off_topic yield. That report is how the query-set design question stops being a judgement call: a query whose share is mostly off_topic is spending guaranteed round-robin slots on noise and should be dropped or rephrased next run (see the VOCABULARY-OWNERSHIP TEST in references/intent-recognition.md). Papers reachable from several queries are credited to each. This runs BEFORE tagging so the per-paper pattern-tag pass only sees survivors, and it **replaces the weak Haiku `outside_taxonomy` tag** as the precision gate. `core` alone feeds the deep-read pool; `adjacent` stays as a citeable baseline/backbone but never consumes a fetch slot. `next` drives the sub-flow; the marker guards re-runs.

Retrieval takes 3-10 min (the openreview connector alone budgets 600s) — set your Bash timeout ≥ 600s or run it in the background.

**Pattern tagging (host step):** classify each `lit_results.json` paper per `references/pattern-summary-rubric.md` into 1-3 of the 15 patterns → write `lit_table.md` with columns `paper_id | year_month | venue | title | ideation pattern tags | bottleneck this paper targets | open issue / unresolved gap | resolves_problem | retrieved_via`. Pure classification — it does not need the large reasoning model: route it to a cheaper/faster model tier or a lower reasoning effort BY DEFAULT (the same tier `NOVELTY_LLM_CLASSIFY_FAST_CMD` names in § Configuration; spending the large model here is pure waste); only when no cheaper tier exists, run it isolated on the host model. Rows are per-paper independent, so for 40+ papers the tagging MAY be sharded across 2-3 parallel fast-tier sub-agents (contiguous slices; assemble with the deterministic `lit_table_merge --out $RUN_DIR/phase0/ --shards <row-file>...` — it validates 9-column shape, row count == paper count, and paper_id coverage against lit_results.json, replacing hand-checks) — wall-clock win, no quality delta.

**Phase 0.5 coverage check (host-recall channel, host step; DEFAULT ON, disable with `IDEASPARK_COVERAGE_CHECK=off`):** after tagging, before fulltext, the host — having read the whole table — names up to 8 clearly load-bearing works the retrieval pool MISSED, writing `phase0/host_refs_nominations.json` `[{title, id_hint?, why, source: parametric|websearch}]`. It PRIORITIZES the last ~12 months (recent/frontier work the dated retrieval windows under-sampled — the target that keeps the diagnosed gap current); older foundational papers (canonical base policies, >12-month landmarks) are mostly Phase 1 lineage's job and should be nominated only as a small minority, and only when a load-bearing backbone/baseline the candidate builds on or is measured against. Use YOUR OWN model here (open-ended intent judgment — do NOT downgrade to a cheap tier); WebSearch is allowed ONLY in this step and ONLY to find titles, never to fabricate a record; an empty `[]` is a valid honest output. Then `add_host_refs --out phase0/ --refs <noms>` (deterministic) VERIFIES each nomination via the SS/arXiv connectors (>=0.9 title match) — verified records merge into `lit_results.json` with `retrieved_via=host_recall|host_web`; unresolvable titles (likely hallucinations) are NOT admitted, they land in `host_refs_unresolved.md`; admitted ids are recorded in `host_refs.json` for provenance. Newly-admitted rows get tagged (fast tier) and `lit_table_merge`d in; `.coverage_check_done` guards against re-running. This keeps `lit_grounding_mode` honestly `real` (every record is connector-verified) while letting host knowledge repair a thin or stale pool. `next` drives the whole sub-flow. Admitted refs feed the fulltext H tier (below).

**Title-named user refs:** if the user query names anchor papers by TITLE ("based on the LoRA paper" — anything the URL/ID regex can't catch), register each BEFORE `phase0_fulltext` via `add_user_ref` (entry-point table). It does a deterministic dedup-merge into `user_refs.json` — do NOT hand-edit that file (some harnesses' file-write tools refuse to overwrite a file that was never read, and a malformed edit silently drops the U fetch tier).

**Phase 0+ full-text fetch — MANDATORY.** The instant `lit_table.md` lands (and after the coverage check), run `phase0_fulltext` (entry-point table) before touching Phase 1. Pool = U (user refs, never capped) + **H (host-recall refs, retrieved_via host_*, cap 6)** + T2 (published on-topic, cap 10) + T3 (arxiv recent on-topic, cap 15; also absorbs unused H/T2 slots), ceiling **25** excluding U, method-first ordering, concurrent fetch (HTML path first, pymupdf PDF fallback; per-paper budget so one slow PDF can't stall the step). **Only `core`-tagged papers are eligible** (Phase 0.4 relevance stamp; U bypasses, H needs its nomination tagged core) — `adjacent` papers stay citeable at abstract level but never consume a deep-read slot. Any pooled paper that fetches to an **empty method section is backfilled** — swapped for the next-ranked core reserve that has one (one-out-one-in, ceiling held; `--backfill` reserve size, default 6). Output `fulltext_cache.json` keyed by paper_id (`{tier, intro, method, source_used, warning}`); fetch failures degrade to abstract + warning. Phase 1 **hard-gates** on this file (`error: fulltext_not_fetched`). Alongside the blob it writes a derived per-paper read view (`phase0/fulltext/index.json` + one `.md` per paper, same content, index carries tier/source_used/warning) — Phase 1 reads the index then only the papers it needs; the blob stays canonical and `phase1_fulltext_topup` refreshes both. Fetches hit a cross-run content cache (`~/.cache/ideaspark/fulltext/`, successful fetches only, 30-day TTL; disable with `IDEASPARK_FETCH_CACHE=off`), so papers recurring across runs on adjacent topics cost zero network time.

### Phase 1 — Bottleneck identification

One isolated LLM call. Prompt: [references/system-prompts/bottleneck_identify.txt](references/system-prompts/bottleneck_identify.txt). Inputs: user query + intake, `phase0/lit_table.md`, `phase0/fulltext_cache.json` (all-failed cache → continue with `fulltext_degraded: true`, abstract-level residue confidence), `phase0/lit_results.json`. Output `phase1/phase1_output.json`: `intake` (+`_inferred_fields[]` — missing fields are inferred, never asked), `bottleneck_statement` (≥2 paper_id cited inline), `closest_adjacent[]` (`{paper_id, summary_and_residue}`), `what_phase_0_did_not_address[]` (each gap ends with an inline stakes clause — practitioner costs and intellectual costs are coequal; gaps are framed as structural properties of a problem class when honestly true, with the anchor as primary instance, else declared class-of-one), `state ∈ {proceed, do_not_generate}`.

Routing: **proceed** (literature-groundable, no OOD trigger) or **do_not_generate** (too-broad / no-anchor OOD, <5 truly-relevant papers, genuinely blank space, or benchmark/system construction) → write `do_not_generate.md` with concrete remedial steps — terminal.

### Phase 2 — Selection + generation (ONE isolated context, TWO outputs)

Run 2.1 and 2.2 back-to-back in one isolated context, writing BOTH output files (they are both generation-side; only adversarial pairs need separate calls):

**2.1** — prompt [references/system-prompts/ideate_select.txt](references/system-prompts/ideate_select.txt); inputs `phase1_output.json`, `references/ideation-patterns/overview.md` (all 15 patterns' Definition / Operational signature / When to apply — selection at WHAT/WHEN level), `references/ideation-patterns/companion-combos.md`, `lit_table.md`. Pick the anchor gap (type-bound to `intake.contribution_type`); commit the PATTERN COMPOSITION — ≥2 distinct patterns by default, realized preferentially as a CHAIN on the anchor (attested second pattern from companion-combos, with a named intermediate object) and/or via a sibling that EARNS its seat through the removal test ("the anchor's story is incomplete without it") — anchor-only is legitimate and common (1-3 gaps total, ≤2 siblings); a single-pattern selection requires a `composition_note` defense (validator presence-checks it; audit weighs it). Record saturation (transparency, not a filter). Output `phase2_select/phase2_select_output.json`: `selected_gaps[]` (index 0 = anchor) + `coherence_thread_type` (or `n_a` when anchor-only) + `composition_note` + `pattern_saturation` + `deferred_gaps[]`. **Retry mode:** when `$RUN_DIR/attempt_1/` exists, the prompt's OPTIONAL retry input applies — the archived audit + selection become negative constraints; anchor-only is a valid retry outcome. **Cross-run dedup:** `next` scans sibling run dirs and appends a soft-negative-anchor input line (titles + signature terms of their canonical candidates, 5 most recent) so adjacent-direction runs don't silently re-invent the same mechanism family; soft by design (see ideate_select.txt's OPTIONAL cross-run input), disable with `IDEASPARK_CROSS_RUN_DEDUP=off`.

**2.2** — prompt [references/system-prompts/ideate_generate.txt](references/system-prompts/ideate_generate.txt); inputs 2.1 output, `phase1_output.json`, `lit_results.json` (closest_adjacent entries ONLY — filter to those paper_ids before reading; the prompt forbids pulling the full dump into context), plus for each gap ONE picked sub-pattern card from `references/ideation-sub-patterns/` (compare `when_to_pick_this_one` + `differentiation_within_parent` via its overview.md; then read the picked card's `tactical_pattern` + Step-by-Step). Output `phase2_generate/phase2_generate_output.json` — ONE candidate, 12 flat fields: `title` / `hook` / `core_mechanism` / `core_mechanism_reasoning` / `core_mechanism_steps`; `gap_closure[]` (`{gap, main_pattern, sub_pattern: "C## (parent pattern name)", how_closed}`, mirrors selected_gaps one-for-one); `falsification_prediction` (single paragraph: minimal experiment + metric-with-direction + ONE named load-bearing variable + negative control on that variable predicting the DOWNSTREAM outcome metric returns to baseline — non-tautological); `compute_budget` (user-relative, GPU-day line + API-dollar line when the campaign calls paid APIs; default intake envelope = 80GB-class GPUs, ≤8 concurrent, ≈150 GPU-days / 5 months, ~$10k API — overridable per user via `IDEASPARK_DEFAULT_COMPUTE`, see § Configuration); `differentiation_from_lit[]` (substantive deltas, not "different pattern"); `almost_prior_paper_id` + `what_step_was_missed`; `signature_terms[]` (own vocabulary — recent collision channel); `alias_terms[]` (other communities' names for the same mechanism, from parametric knowledge — multi-year alias collision channel); `composition_note` (echoed verbatim from 2.1). Reality constraints at writing time: `core_mechanism_reasoning` carries three mandatory blocks — a PREMISES ledger (load-bearing premises + domain justification; bets tagged `untested — falsification target`; WHEN the mechanism consumes sampled/observed data, at least one OBSERVATION-MODEL premise — how are the inputs sampled, is that unbiased here — else the literal `observation-model: n/a`; WHEN the mechanism estimates any quantity, declare the ESTIMAND precisely), a NAIVE-BASELINE AUDIT (state the naive version; why doesn't it already work — branch (i) naive relies on a false premise → confronting it IS the contribution, with a STANDARD-TOOL FOLLOW-UP: if the confrontation is a textbook tool from another field, name the domain-specific structure that makes this instance unsolved or declare the contribution application-grade, (ii) naive suffices → incremental signal surfaced honestly, (iii) naive works but the field disbelieves → minimalism with evidence), then design rationale — plus every invoked dataset / model-access level / annotation / tool must be a nameable EXISTING artifact (hard rule 3; modest self-built resources allowed with cost counted in compute_budget) and every claim stated at its defensible strength — guarantee-grade wording only with assumptions stated where it appears (hard rule 4; the coherence gate grades this, and a strong claim with honest assumptions beats a hedged weak one) — and every NUMBER carrying provenance (hard rule 5): method parameters are named symbols with a default + selection rule (a bare `>= 95%` / `top 5` / `every 2e3` is forbidden — an unnamed quantity cannot be swept or graded), while `falsification_prediction` states metric + direction + control and may carry a numeric bar ONLY when tagged `derived:` or `measured in <paper_id>`; invented bars are fabrication. Artifact versions, algebraic constants, definitional settings and experiment-scale counts are not parameters and are unaffected. Both kill-switch fields (`falsification_prediction`, `compute_budget`) are locked from here on — see Phase 3 for the single audited exception.

**Citation gate (deterministic, MANDATORY before Phase 3):**

```bash
python3 "$SKILL_DIR/scripts/run.py" validate --phase2 $RUN_DIR/phase2_generate/phase2_generate_output.json
```

Any `fail` = a `sub_pattern` citation was guessed from the parent's gist, not read from `overview.md`. Fix against `references/ideation-sub-patterns/overview.md` (or regenerate 2.2 with the card open) and re-run until clean — the gate proves parent-consistency only; whether core_mechanism performs the cluster's actual tactic is Phase 3.2's `recipe_application_check`. (`next` runs this gate automatically.)

**Coherence gate (2.3 — one isolated LLM call, MANDATORY after the citation gate, before the 3.2 audit; 3.1 collision may run CONCURRENTLY with it):** prompt [references/system-prompts/coherence_trace.txt](references/system-prompts/coherence_trace.txt); inputs: the 2.2 candidate + the 2.1 spec; MUST be a FRESH context, never the 2.1+2.2 agent (the context that wrote a logic bug rubber-stamps it). It verifies internal procedural validity by EXECUTION, not review — five trace actions: formalize the dataflow (undefined symbols, missing producers, circular deps, and UNBOUND PARAMETERS — a number the procedure reads as a parameter but that carries no name/default/selection rule, the form a magic constant takes to evade every symbol-keyed check; method fields only, `falsification_prediction` numbers belong to the audit), numeric dry-run on one small concrete instance (magnitude/probability absurdities — logic bugs read fluently and only surface when computed; when a code-execution tool exists the gate WRITES AND RUNS a stdlib Python script and pastes script+output, else hand-computes marked `unexecuted`), degenerate probes (empty/k=0/ties), claim→step mapping AND grading (every asserted property mapped to the step that establishes it AND graded `established` / `conditional` — assumptions listed / `overclaim` — wording downgraded / `empirical`; statistical claims settled by an executed Monte Carlo with the measured number, theorem-shaped claims recorded as proof obligations — at best conditional; grading never rewards vagueness: all-hedged claims are themselves a weakness finding), and a NAIVE-BASELINE COMPARISON (independently construct the naive version — never the candidate's own stated one — run it on the same instance, and judge against the candidate's declared branch: `confronts_obstacle` / `equivalent_to_naive` / `n_a`). OBSTACLE HOLES — findings whose honest fix is a redesign, or which coincide with the obstacle the candidate declares it exists to solve (incl. `equivalent_to_naive`) — must NOT be patched around with avoidance-style repairs (abstain/clamp/skip); they go to `unrepaired[]` as blocking, verbatim. Verdict `pass` | `patched`; repairs are patch-only via the SAME merger (`--out-name refined_candidate.json`), scoped to making the written procedure sound (core_mechanism*, how_closed narrative, signature/alias terms when the repair changed what the mechanism is) — novelty surface, pattern bindings, and kill-switch fields are out of scope; unfixable-without-redesign findings go to `unrepaired[]` for the audit to weigh. Single pass, never abandons. The 3.2 audit stays blind to this report's trace/verdict/patches — with ONE exception: blocking `unrepaired[]` entries are executed evidence, so the gate ALSO writes them to `phase2_coherence/blocking_findings.json` (self-contained: verbatim step quote + script excerpt + measured numbers + reading_dependence + `structural_requirement` per entry — what it would TAKE to pass this obstacle: the identifying condition any mechanism here needs, which sources of it are available vs excluded and why, and what is worth salvaging; diagnosis, not the redesign the repair contract forbids, and honestly empty when the evidence supports nothing concrete — `next` passes it to the retry as a directive), which `next` lists as a 3.2 input; the audit must disposition each (uphold, or refute by naming a concrete modeling/arithmetic flaw — `next` deterministically bounces an audit report that skipped dispositions or advanced over an upheld finding). Formalization ambiguity is controlled by the gate's dual-reading protocol (every defensible reading of an ambiguous operative term is executed; findings are tagged reading_robust vs reading_dependent). When `refined_candidate.json` exists it is the canonical candidate for every later phase (`next` wires this automatically). It validates that the algorithm survives on paper — NOT that it works empirically (falsification experiment) or is novel (audit).

### Phase 3 — Quality gauntlet

**3.1 collision (orchestrator, no LLM):** entry-point table. May be LAUNCHED IN PARALLEL with the 2.3 gate (on the 2.2 output — `next` emits the background launch alongside the 2.3 step): collision reads only `signature_terms[]`/`alias_terms[]`, a `.collision_terms.json` sidecar records the terms actually used, and `next` re-issues collision if a coherence patch changed those terms (rare — term repairs are scoped to mechanism-changing patches). TWO retrieval channels over all 4 connectors, merged into `collision_hits.json` with a per-hit `collision_channel` tag: **signature** — the candidate's `signature_terms[]` over a 10-month window (contemporaneous scoop risk); **alias** — the candidate's `alias_terms[]` (other communities' names for the same mechanism, produced from parametric knowledge at 2.2) over a 48-month window (renamed-ancestor risk — the "goal-conditioned success detector vs goal-image conditioned scorer" blind spot is lexical, not temporal, so widening the signature window alone cannot catch it). Missing `signature_terms[]` → rc=11 sentinel: produce BOTH term sets per intent-recognition.md Collision mode (terms 3-7 words each — long sentences break URL encoding), edit the candidate JSON, re-invoke. Missing only `alias_terms[]` → loud warning, alias channel skipped (add the field and re-run to close the blind spot). The audit-facing pool is relevance-truncated per channel (≤120 hits/channel by lexical overlap with the channel's own terms; zero-relevance BM25 noise dropped unconditionally; drops printed; untruncated pool preserved as `collision_hits.full.json`), so the audit can consume `collision_hits.json` in a few sequential Read chunks — no jq two-pass triage needed.

**3.2 audit (one isolated LLM call):** prompt [references/system-prompts/critique.txt](references/system-prompts/critique.txt); inputs: candidate, 2.1 spec, `lit_table.md`, `collision_hits.json`, `references/anti-patterns.md`, `phase2_coherence/blocking_findings.json` when it exists (the 2.3 gate's executed blocking evidence — the report must disposition each entry, and advance is forbidden while one is upheld; a `refuted` disposition additionally triggers a bounded refutation re-check (`refutation_recheck.txt`, fresh call) that `next` requires before trusting the verdict — an invalid refutation counts as upheld and bounces the audit), and each cited sub-pattern card `references/ideation-sub-patterns/<C##>.md` (strip the leading code from `sub_pattern`; typically 1-3 cards, others NOT loaded). Five corpus-anchored checks:

| Check | Question |
|---|---|
| gap_closure_reject_check | does the candidate match a documented Reject lesson in each cited sub-pattern card (`## Tactical failure mode` + ALL `### Reject lessons` bullets)? |
| recipe_application_check | does `core_mechanism` actually perform the cited C## cluster's `## Tactical pattern` signature move, or only the parent's generic idea (`bypassed` — the leading cause of incremental output)? |
| anti_pattern_check | if the SET of `gap_closure[].main_pattern` matches a reject-favored composition, is the required mitigation substantively delivered (artifact, not keyword)? |
| paper_pointed_threat | most specific subsuming/competing paper in `lit_table ∪ collision_hits` (both channels; alias-channel threats are NOT discounted for age); `no_threat_found` is valid — fabricating a generic threat is forbidden. Side output `parametric_family_concern`: a named un-retrieved mechanism family from parametric know

…(truncated)