Idea

Idea

Use this skill to turn the current baseline and problem frame into concrete, literature-grounded, frontier-aware, testable directions. The goal is to choose the next executable research route, not to maximize brainstorming volume or reward shallow novelty.

When startup_contract.need_research_paper = false and the quest already has a concrete optimization handle, idea may stop after selecting or seeding a direction and then hand off into optimize instead of insisting on the full paper-oriented ideation loop. In that algorithm-first case, idea should usually produce a small method-brief frontier and then defer candidate ranking, promotion, and bounded search to optimize. When doing that handoff, prefer the brief-shaping discipline later used by optimize: clarify the bottleneck and constraints, keep only a small differentiated 2-3 option slate, and hand off a recommended brief rather than a pile of loose intuitions.

Match signals

Use idea when:

• the accepted baseline and metric contract already exist, but the next route is still unresolved
• the current line failed and the quest needs a new falsifiable direction
• the problem is not "build a new module" but "decide what kind of route should be tried next"
• the current bottleneck might be a mechanism problem, an objective mismatch, a measurement/evaluator problem, or an infrastructure constraint that changes what should be tested next

Do not use idea when:

• the baseline gate is still unresolved
• the current board state is too stale or conflicting to say what the mainline actually is
• the next step is already obviously write, review, or finalize

If the current board cannot be compressed cleanly, route through decision or intake-audit before widening the frontier.

One-sentence summary

Turn the current objective, board state, and bottleneck into a small differentiated frontier, then select the next falsifiable route.

Control workflow

1. Write the objective contract. Use references/objective-contract-template.md. Make the real target, trusted proxies, false-progress signals, and hard constraints explicit before generating ideas. Default durable path: artifacts/idea/objective_contract.md.
2. Write the current board packet. Use references/current-board-packet-template.md. Compress the incumbent, latest decisive result, active blocker, and stale routes-to-ignore into one current state surface. Default durable path: artifacts/idea/current_board_packet.md.
3. Identify the important contradiction and plausible novelty source. Use references/high-value-idea-sourcing.md. Start from the most important unresolved contradiction, anomaly, bottleneck, or failure region rather than from a preferred mechanism.
4. Run a broad, history-aware literature search before proposing serious ideas. Use references/related-work-playbook.md, references/research-history-playbook.md, and references/literature-survey-template.md. Cover direct in-domain frontier papers, foundational papers, strongest nearby competitors, and cross-domain papers whose mechanisms may translate into the current task. If the runtime prompt explicitly enables cross-quest recall, follow that injected policy before going outside; otherwise stay inside the current quest's memory/artifacts and explicit user-provided files. See playbook §2.1 for the full source-order protocol. If DeepXiv is available, use it for broad paper-centric discovery and citation expansion; otherwise use search engines and citation chaining directly. Do not promote or even seriously shortlist a new idea until the durable survey and closest-prior-work comparison are updated enough to judge novelty and feasibility honestly.
5. Extract the limitation pattern and novelty opportunity from the survey. Distinguish what is already saturated, what is only a decorative tweak, and what could still support a differentiated route. Default against small local edits unless they are explicitly shown to be the highest-value surviving route.
6. Choose the idea family mix. At minimum, decide whether the current pass should consider some mix of:
   • mechanism-family routes
   • objective-family routes
   • measurement-family routes
   • infrastructure-family routes
7. Run bounded brainstorming. Use references/controlled-brainstorming-playbook.md when the route is not already obvious. Generate a small, meaningfully different slate rather than a pile of micro-variants. Prefer candidate families that could change the conclusion, capability boundary, or paper value materially, not just move a knob.
8. Write a compact pre-idea draft for the serious surviving candidates. Use references/pre-idea-draft-template.md. Normally write drafts for the top 1-3 candidates, not for the whole raw slate. The draft must surface hidden assumptions, local-optimum lock-in risk, strongest outside-family alternative, strongest rejection case, and the cheapest falsification path before any formal idea submission. Default durable path per candidate: artifacts/idea/pre_idea_drafts/<candidate_id>.md.
9. Filter aggressively. Use references/selection-gate.md. Remove candidates that only improve a surrogate, reopen a stale route without new evidence, violate leakage or submission-time boundaries, or lack a cheap falsification path.
10. Select and hand off. The selected package must include the route, why now, novelty type, main risk, anti-win condition, core hypothesis, mechanism sketch, strongest falsification experiment, minimal validation, abandonment condition, and the next stage.

Draft-before-submit SOP

Before a direction is formally submitted as the selected idea, write a compact pre-idea draft or equivalent durable challenge memo for each serious surviving candidate.

The default rule is:

• raw brainstorming can widen the frontier
• pre-idea drafts narrow and stress-test the frontier
• only then can a final selected idea be submitted

The pre-idea draft exists to stop three failure modes:

• local-optimum lock-in around the current mainline
• hidden assumptions staying implicit
• attractive ideas being promoted before the strongest rejection case is written down

Unless there is already an up-to-date equivalent artifact, do not formally submit the final idea until at least one pre-idea draft has:

• been written for the likely winner
• been compared against the incumbent and at least one outside-family or assumption-reversal alternative
• been revised, rejected, or promoted based on that comparison

Default durable path rules:

• objective contract: artifacts/idea/objective_contract.md
• current board packet: artifacts/idea/current_board_packet.md
• candidate frontier summary: artifacts/idea/candidates.md
• pre-idea draft per serious candidate: artifacts/idea/pre_idea_drafts/<candidate_id>.md
• final selected idea: artifacts/idea/selected_idea.md

When a candidate is promoted, artifacts/idea/selected_idea.md should point back to the winning pre-idea draft path instead of losing that lineage in prose only.

AVOID / pitfalls

• Do not start from swapping method A for method B before naming the important contradiction or bottleneck.
• Do not brainstorm before the real objective and false-progress signals are explicit.
• Do not treat lower loss, better average surrogate, or cleaner intermediate metrics as route health if the real target is unchanged.
• Do not reopen stale routes unless new evidence explicitly weakens the current mainline.
• Do not generate a large within-family variant swarm before the mechanism family itself is chosen.
• Do not propose an idea as "new" before the direct-field and adjacent transferable literature have both been checked.
• Do not default to cosmetic modifications, parameter nudges, or tiny architecture swaps unless the survey and bottleneck analysis show they are genuinely the best surviving route.
• Do not let the current mainline, favorite mechanism, or easiest implementation path lock the search into a local optimum while the hidden assumptions behind that line remain unchallenged.
• Do not jump from brainstorming notes straight to final idea submission without a compact draft that forces the hidden assumptions, strongest rejection case, and falsification path into the open.
• Do not treat novelty as “totally unprecedented”; it may come from a new problem, view, mechanism, method, setting, evaluation, or boundary condition.
• Do not promote a direction that fails a value/feasibility screen simply because it sounds exciting.
• Do not promote a direction without a cheap falsification path and a visible anti-win condition.

Constraints

• Keep the accepted dataset, metric, and evaluation contract fixed unless scope explicitly changed.
• Do not propose routes that depend on submit-time unavailable features.
• Do not propose routes that introduce leakage-prone targets or labels into training.
• Do not let implementation convenience outrank target alignment.
• Search should be broad enough to map the main paradigms, history, and strongest overlaps, not just skim a few recent papers.
• Search must cover both the current field's strongest direct papers and adjacent or cross-domain papers whose mechanisms may translate into the current task, evaluator, or systems setting.
• If DeepXiv is available, prefer it for broad paper-centric discovery; otherwise use search engines, citation chaining, and open-web search directly.
• Serious idea generation should happen only after the survey is broad enough to rule out obvious rediscoveries and to reveal non-trivial opportunity gaps.
• A serious candidate should be explainable in terms of importance, novelty type, feasibility, verification path, and failure value.
• By default, prefer routes with step-change or boundary-changing potential over small local refinements.
• Small refinements are allowed only when the literature and current evidence indicate they are still the highest-value route and the reason is stated explicitly.
• In system optimization work, a valid idea may be a mechanism change, an objective/evaluator correction, a measurement fix, or an infrastructure change if that is what best improves the real target.

Validation

Before the idea pass can end, the durable selected idea package should make explicit:

• the important contradiction, gap, anomaly, or bottleneck it is targeting
• the literature coverage used to justify the route, including direct-field papers and adjacent transferable papers
• the dominant novelty type
• the targeted limitation
• the real objective and the strongest false-progress signal
• the pre-idea draft or equivalent challenge memo that preceded promotion
• the selected direction and why it won now
• why the selected route is not merely a decorative tweak relative to the closest prior work or current baseline
• the value/feasibility screen or equivalent judgment
• the core hypothesis
• the mechanism sketch
• the strongest falsification experiment
• the anti-win condition
• the minimal validation
• the abandonment condition
• the next stage

If those fields are still fuzzy, continue ideation or route back through decision rather than pretending the route is ready.

Interaction discipline

• Follow the shared interaction contract injected by the system prompt.
• For ordinary active work, prefer a concise progress update once work has crossed roughly 6 tool calls with a human-meaningful delta, and do not drift beyond roughly 12 tool calls or about 8 minutes without a user-visible update.
• Keep ordinary subtask completions concise. When the idea stage actually finishes a meaningful deliverable such as a selected idea package, a rejected-ideas summary, or a route-shaping ideation checkpoint, upgrade to a richer artifact.interact(kind='milestone', reply_mode='threaded', ...) report.
• That richer idea-stage milestone report should normally cover: the final selected or rejected direction, why it won or lost, the main remaining risk, and the exact recommended next stage or experiment.
• That richer milestone report is still normally non-blocking. If the next experiment or route is already clear from durable evidence, continue automatically after reporting instead of waiting.
• If the runtime starts an auto-continue turn with no new user message, keep advancing from the active requirements and current durable state instead of re-answering the previous user turn.
• Message templates are references only. Adapt to the actual context and vary wording so updates feel natural and non-robotic.
• If a threaded user reply arrives, interpret it relative to the latest idea progress update before assuming the task changed completely.

Three-layer todo contract

• keep quest-root plan.md as the research map for the whole quest loop
• keep workspace PLAN.md as the active idea-node contract when ideation is multi-step, literature-heavy, or route-sensitive
• keep workspace CHECKLIST.md as the active ideation frontier with one real in-progress item and a short Next list
• if the execution frontier stops changing across repeated passes, revise the node contract or the research map instead of nesting more substeps

Research-map role

• idea selects or refreshes the next route within the current loop; it does not replace the whole quest roadmap
• when an idea is selected, rejected, or downgraded, update quest-root plan.md so the next experiment node or fallback decision node is explicit
• when a strong result later becomes the new incumbent, the next idea pass should open a new loop entry in quest-root plan.md rather than drifting into ad hoc brainstorming

Current-node plan and checklist

When ideation becomes multi-step, create or refresh:

• workspace PLAN.md as the current idea-node contract
• workspace CHECKLIST.md as the ideation frontier

The idea node should make explicit:

• which bottleneck is being attacked now
• which candidate families are still live
• what selection gate must be cleared before experiment

Before widening the frontier, the node should also make explicit:

• the current objective contract
• the current board packet
• which candidate-family mix is actually being explored in this pass

Stage purpose

The idea stage should not generate vague inspiration. It should produce executable hypotheses tied to:

• the active baseline
• the current codebase
• the accepted evaluation contract
• the strongest relevant prior work

This stage is not just "brainstorming". It is a controlled brainstorming plus route-selection stage. It still needs a bounded creative-divergence phase before convergence. Do not collapse onto the first plausible route just because it sounds implementable. Do not settle for a low-amplitude tweak when a broader, still-feasible route remains live. It should normally create a new candidate direction branch and node; it does not by itself decide the next optimization round. The output must survive three checks at once:

• novelty or at least clear research value
• feasibility in the current repo and resource budget
• manuscript defensibility if the line later becomes a paper claim

When multiple routes survive, prefer the most differentiated route that is still falsifiable and executable in the current repo, rather than the easiest tiny patch.

When the route already looks likely to become a paper-facing line, seed one lightweight structured outline candidate during idea work. Use artifact.submit_paper_outline(mode='candidate', ...) for that seed instead of leaving the future paper structure only in prose. Use references/outline-seeding-example.md for the minimum acceptable shape. The idea-stage outline candidate is not the full paper line yet, but it should already name the likely one-sentence paper idea, scoped claims, research_questions, experimental_designs, and the first section-level evidence needs that later supplementary slices must satisfy. Keep that seed minimal and executable: a short paper_view plus expected evidence items is better than a long narrative outline with no concrete evidence hooks. If the current research head, strongest measured branch, or active runtime refs are unclear after resume, call artifact.get_quest_state(detail='summary') and artifact.list_research_branches(...) before choosing a foundation. If the current brief / plan / status wording matters for direction choice, call artifact.read_quest_documents(...). If earlier user conversation materially changes the direction-selection target, call artifact.get_conversation_context(...) before locking the next idea.

Finishing one idea deliverable is not quest completion. After reporting a completed idea package, continue into the next justified stage unless a real blocking decision is still unresolved.

When the quest disables research-paper delivery, keep manuscript defensibility secondary to:

• algorithmic value
• feasibility
• clean experimental follow-through
• durable recording of why this direction should be the next measured attempt

Before starting a genuinely new round, default to the current research head as the foundation. However, you may deliberately choose a different foundation when the durable evidence says it is better. When the best starting point is not obvious, inspect artifact.list_research_branches(...) first and compare:

• current head
• baseline foundation
• strongest recent measured branch
• older but cleaner branch

If you do not use the default current head, record the reason explicitly in the new idea submission. Treat a newly accepted branch as one durable research round. If the active branch already has a durable main-experiment result and you are starting a genuinely new optimization round, prefer creating a child branch from the chosen foundation rather than revising the old branch in place.

At the direction level, prefer elegant algorithmic or theoretical improvements over brute-force cost-for-performance tradeoffs whenever possible.

This stage should preserve the strongest old DeepScientist direction-selection logic:

• understand the baseline and its failure modes
• search related work broadly before claiming an idea is good, including adjacent fields with translatable mechanisms
• derive limitations
• produce a compact set of candidate ideas from an explicit direction set
• rank them with explicit tradeoffs
• choose a direction with a clear evidence-based decision path
• ensure the selected direction is manuscript-defensible rather than merely implementation-plausible

Use a compact search discipline during ideation:

• first identify the current strongest line from existing results, literature, and branch history
• treat that line as the current incumbent
• keep only a small serious frontier, usually 2-3 serious alternatives and rarely more than 5 after one bounded widening pass
• ensure the frontier is meaningfully differentiated rather than the same idea renamed
• prefer selecting from existing evidence over expanding the candidate list indefinitely

Candidate sets should usually cover some mix of:

• a strong local refinement of the incumbent
• an orthogonal alternative that addresses the same bottleneck differently
• a cleaner or more defensible route with lower conceptual complexity
• an objective/evaluator fix when the current route may be optimizing the wrong thing
• an infrastructure or throughput fix when measurement cost itself is blocking useful iteration

Do not default to “run a small experiment and see” as the way to break ties. Break ties primarily through careful reasoning over:

• existing experiment results
• failure patterns
• related-work overlap
• code-path feasibility
• claim defensibility

Non-negotiable rules

• Do not claim novelty without a written related-work comparison.
• Do not select an idea before checking whether close prior work already did it.
• Do not confuse "I can implement this" with "this is a publishable or useful research direction".
• Do not treat a weak literature search as sufficient because the idea sounds elegant.
• Do not start serious ideation from memory or taste alone; refresh the external literature unless the existing survey already covers the needed frontier and that reuse is recorded explicitly.
• Do not treat the current field as the only search space; check cross-domain mechanism transfer whenever the bottleneck might admit it.
• Do not promote a small tweak by default; if an incremental route wins, record why broader routes failed on novelty, feasibility, or claim value.
• For paper-ready idea packages, aim for a durable survey that usually covers at least 5 and often 5-10 task-modeling-related, mechanism-relevant, or otherwise directly usable papers.
• If the direct task-modeling neighborhood truly contains fewer than 5 usable papers, record that evidence explicitly and fill the remaining coverage with the closest adjacent papers whose mechanism can still be translated into the current task and codebase.
• Algorithm-first exception:
  • when startup_contract.need_research_paper = false and a concrete optimization handle already exists, you may stop after a memory sweep plus a small targeted paper check instead of satisfying the full 5-10 paper floor
  • use that exception only when the immediate goal is method-brief selection for optimize, not paper-level novelty claims
  • if you use the exception, say explicitly that the output is an optimization brief frontier rather than a paper-ready idea package
  • still shape that frontier deliberately: clarify the bottleneck and comparability boundary first, keep a differentiated 2-3 candidate slate, and explain why one brief is recommended now
• Every fresh idea build or idea-refinement pass should begin with:
  • a memory sweep, and
  • an external literature sweep or a clear reason why the existing survey is already sufficient.
• For paper-ready promotion, refresh artifacts/idea/literature_survey.md or an equivalent durable survey report before the direction is promoted.
• Every survey update must explicitly separate:
  • reused prior survey coverage
  • newly added papers or comparisons from this pass
  • still-missing or unresolved overlaps
• When a web/search tool is available, actively use it. Prefer web search for paper discovery, usually targeting arXiv first, then expand with citation and open-web search for neighborhood coverage.
• If DeepXiv is declared available by the system prompt, prefer the DeepXiv route for paper-centric discovery and shortlist paper triage before broad open-web search.
• If DeepXiv is declared unavailable, do not try to force it; stay on the legacy route.
• When a concrete arXiv paper needs to be read, compared, or summarized, use artifact.arxiv(paper_id=..., full_text=False). Keep search in web discovery by default; use artifact.arxiv(...) for reading shortlisted papers, and set full_text=True only when needed.
• Before opening a broad new search, check quest and global memory with memory.search(...) and reuse existing paper notes, idea notes, and knowledge cards.
• Search for genuinely missing, newly relevant, or more recent papers whenever possible. Do not rerun the same broad search without stating what gap the new search is meant to close.
• Do not introduce a new dataset or a new evaluation regime unless the quest scope explicitly changed.
• Do not rely on human evaluation or subjective assessment for idea validation; the eventual experiment must remain automatable with code and accepted metrics.
• Treat ideation as read-heavy and write-light: inspect code and papers, but avoid substantial implementation during this stage.
• Do not propose directions that require new datasets.
• Do not default to brute-force engineering escalation when a cleaner first-principles direction is available.
• Do not keep generating more ideas once a small, clearly ranked frontier already exists.
• Do not treat superficial variation as a new idea if the expected mechanism and evidence burden are effectively unchanged.
• Separate generation from evaluation during ideation: generate first, judge second.
• Start each fresh ideation pass by classifying the current framing as problem-first or solution-first.
• Unless strong durable evidence already narrows the route to one obvious serious option, run one bounded divergent pass that produces a small but meaningfully varied slate, usually 6-12 raw ideas before collapsing to a serious frontier that is usually 2-3 and at most 5.
• If all surviving candidates belong to the same mechanism family, widen once with at least two new ideation lenses before converging.
• Keep structurally coherent rejected ideas in a parking-lot or rejected-candidate section so they can be recombined later if needed.
• In algorithm-first work, idea should usually produce direction families, not a large within-family variant swarm.
• Treat within-family micro-variants as optimize brief work unless the mechanism family itself is still unresolved.
• Every serious candidate must answer why now? or what changed?, not just what is the mechanism?
• Every selected idea must survive a two-sentence pitch and strongest-objection check before promotion.
• Do not promote a direction unless you can explain:
  • what limitation it targets
  • why prior methods do not already solve it
  • what evidence would later be needed to defend the claim
• When the likely next route is a paper-facing main experiment plus analysis package, do not stop at prose-only idea notes; seed the likely research_questions, experimental_designs, and per-section evidence needs in the outline candidate.
• If the likely route already has a clear paper-facing structure, seed the future paper line early:
  • identify the likely main-text sections
  • identify which sections will need supplementary evidence rather than only the main run
  • identify the concrete evidence items that must later be maintained in the paper line's outline folder or compiled outline contract
• If the idea is not novel but still worth doing, state that honestly as:
  • replication value
  • transfer-to-new-setting value
  • stronger evidence on an unresolved question
  • negative-result value
  • infrastructure/platform value

Use when

• the baseline is ready
• the task and metric contract are already clear
• the quest needs a concrete research direction
• the current idea line failed and a new direction is needed

Do not use when

• the baseline gate is unresolved
• the quest still lacks basic problem framing
• the next step is obviously a write-up or finalization rather than ideation

Preconditions and gate

Before ideation, confirm:

• there is an active or accepted baseline
• the dataset and metric contract are explicit
• the relevant code path and papers are available
• the strongest obvious related-work cluster can be searched from available references and tools

If these are still unclear, route back to baseline or scout.

Companion skill rule

idea is the anchor skill for direction selection. However, when the quest still needs literature grounding or novelty checking, actively open scout as a companion skill before final idea selection.

In practice:

• use scout to expand the paper set, search adjacent methods, and clarify the baseline landscape
• use idea to convert that landscape into limitations, candidate directions, and a selected idea

Do not skip the scout pass just because the quest is already in the idea stage.

Direction-shaping protocol

Use references/idea-thinking-flow.md when the main need is better reasoning hygiene. Use references/idea-generation-playbook.md when the main need is to create a new idea slate and select one clear next research object. Use references/high-value-idea-sourcing.md when the main need is to identify a truly important contradiction or bottleneck before widening.

Default creation flow for a fresh idea pass:

1. frame one concrete limitation
2. separate symptom / mechanism hypothesis / consequence
3. keep one main hypothesis plus 2-3 competing hypotheses
4. name the primary lever bucket
5. generate a bounded candidate slate from that framing
6. record selected / deferred / rejected outcomes explicitly

Set the frontier width with a validation-cost estimate before widening:

• fast-check: the first objective validation loop is likely under about 20 minutes
• slow-check: the first objective validation loop is likely over about 20 minutes or otherwise expensive in compute, queue time, or human delay

For fast-check idea work:

• allow a slightly wider serious slate when the candidates are meaningfully different
• prefer candidates with cheap, orthogonal falsification paths
• keep more alternatives alive into optimize because validation is cheaper than overthinking

For slow-check idea work:

• keep the serious slate tighter, usually 1-3
• demand a clearer bottleneck story and stronger evidence before adding another family
• prefer the route with the best expected evidence-per-run, not the route with the most speculative upside
• do not hand off a broad speculative slate just because it sounds interesting

Do not start by shopping for modules to add. Do not let one attractive mechanism become the de facto framing before the limitation is pinned down. Do not let direction-family ideation collapse into within-family variant generation too early.

In normal idea work, stop at the direction-family level:

• select which mechanism families deserve serious consideration
• identify the strongest one to carry forward
• hand off within-family brief shaping to optimize when the quest is algorithm-first

If the task still requires choosing among mechanism families, stay in idea. If the family is already chosen and the next need is branchless method-brief shaping, hand off to optimize.

Truth sources

Use:

• baseline artifacts and verification notes
• baseline paper and source repo
• current codebase and recent diffs
• scout notes and paper memory cards
• prior failed runs and decisions
• current task constraints
• quest and global memory cards returned by memory.list_recent(...) and memory.search(...)
• prior literature survey reports and related-work artifacts
• web-search discovery results for arXiv and related sources
• paper-reading notes produced after using artifact.arxiv(...)
• citation trails and open-web search results for nearby work
• citation trails from the baseline paper and strongest nearby papers
• recent papers that share the same task, metric, dataset, mechanism, or bottleneck

Do not rank ideas on style alone. Rank them on evidence, feasibility, and testability.

Related-work and novelty mandate

Before you choose a direction, perform a broad but bounded literature sweep.

The sweep must be grounded in actual retrieval, not recall alone. If durable quest memory already contains a recent and explicit survey, reuse it first and search externally only for the missing buckets, newer papers, or unresolved overlaps. For a normal selected-idea decision, the durable sweep must end with at least 5 and usually 5-10 papers that are close enough to the task-modeling problem, failure mode, mechanism, or codebase translation question to inform the actual design. This floor exists to prevent thin novelty claims and under-motivated ideas, not to reward quota chasing.

Do not treat “recent papers” as a substitute for “the field history”. At minimum, map:

• seminal or foundational papers
• turning-point or paradigm-shift papers
• current mainstream or SOTA papers

Then use citation chaining to reconstruct how the question evolved and where the real breakpoints still are.

When tools allow it, combine:

• memory.search(...) and recent memory reads
• DeepXiv for broad paper-centric discovery and citation expansion when available
• otherwise web search for arXiv and adjacent sources
• artifact.arxiv(paper_id=..., full_text=False) for actually reading shortlisted papers
• citation expansion or open-web search for follow-up papers, code, and comparisons

The sweep should cover at least these search angles:

• direct same-task / same-dataset / same-metric competitors
• methods using the same mechanism or main lever you are considering
• papers targeting the same failure mode or bottleneck
• strong recent papers that may have closed the gap already

When the direct neighborhood looks saturated or too incremental, extend the sweep to adjacent conceptual neighborhoods:

• optimization methods targeting the same instability or objective mismatch
• representation-learning methods targeting the same information bottleneck
• signal-processing, geometry, probabilistic, or control-inspired methods addressing an analogous failure mode
• methods from neighboring tasks that solve the same structural problem under a different surface form

The point is principled translation, not superficial import. Borrow the core mechanism or mathematical idea only if you can explain why it should survive translation into the current codebase and metric contract.

For each promising idea, you must be able to answer:

• which papers are the closest prior art?
• what exactly is the overlap with your proposed mechanism?
• what is still missing, weak, or untested in those papers?
• if they already did most of it, why is this still worth pursuing?

The goal is not to cite everything on Earth. The goal is to avoid fake novelty and to identify a direction that has credible research value. However, do not stop the sweep early once the first plausible argument appears. Keep going until the strongest obvious overlaps are mapped and the 5-10 usable-paper floor is durably satisfied.

Recommended search outputs:

• a compact related-work map
• a closest-prior-work table
• a novelty / value verdict for each serious candidate
• a paper bucket split:
  • core papers
  • closest competitors
  • adjacent inspirations
  • watchlist / uncertain relevance

For a more detailed search and triage method, read references/related-work-playbook.md.

If the search is still too thin to support a novelty or value judgment, the idea stage is not ready to end.

Required durable outputs

The idea stage should usually leave behind:

• an objective contract
• a current board packet
• a limitations analysis
• a literature survey report
• a survey-delta section that marks:
  • reused findings
  • newly retrieved papers this pass
  • unresolved gaps or watchlist items
• a related-work map
• a novelty and research-value audit
• 2-5 candidate ideas, with the final serious frontier usually narrowed to 2-3
• a selected idea or explicit rejection of the current line
• a durable Markdown idea draft that is finalized before the accepted idea is submitted
• one pre-idea draft per serious surviving candidate, usually 1-3
• one or more memory cards for reusable rationale
• one or more quest papers cards for the strongest papers or search clusters
• an idea artifact and a decision artifact

Recommended durable intermediate outputs:

• an outline-style direction note with:
  • executive summary
  • current baseline results and metric direction
  • codebase analysis
  • dataset analysis
  • mathematical problem formulation
  • baseline methods as special cases
  • five actionable research directions
  • evaluation metrics and success criteria
  • infrastructure and constraint notes
  • claim boundary

When producing a fuller research-outline style note, prefer a direct-agent-like structure:

• Executive Summary
• Codebase Analysis
• Limitations / Bottlenecks
• KPIs
• Research Directions
• Risks & Mitigations

Do not force this structure for every tiny ideation turn, but use it when the quest needs a serious research-plan artifact.

Recommended durable files:

• artifacts/idea/objective_contract.md
• artifacts/idea/current_board_packet.md
• artifacts/idea/literature_survey.md
• artifacts/idea/related_work.md
• artifacts/idea/limitations.md
• artifacts/idea/candidates.md
• artifacts/idea/pre_idea_drafts/<candidate_id>.md
• artifacts/idea/selected_idea.md
• artifacts/idea/research_outline.md

When producing the literature survey report, prefer the structure in references/literature-survey-template.md. When writing the objective contract, prefer references/objective-contract-template.md. When writing the current board packet, prefer references/current-board-packet-template.md. When the route needs a bounded but real creative-divergence pass, prefer references/controlled-brainstorming-playbook.md.

When producing a full research-outline style note, prefer the detailed structure in references/research-outline-template.md.

When the runtime supports durable knowledge cards, also preserve:

• incident or failure-pattern lookups relevant to the mechanism
• a reusable knowledge card for the selected idea hypothesis

Thinking protocol

Use the old PI discipline here too. Your analysis should be:

• hypothesis-driven: viewpoint first, evidence second
• pyramid-shaped: conclusion first, then reasons, then action
• MECE where possible:
  • data
  • model
  • objective
  • optimization or training dynamics
  • inference
  • evaluation protocol
  • infrastructure
• SCQA-compatible:
  • situation
  • complication
  • research question
  • answer hypothesis plus 2-3 competing hypotheses

Do not dump disconnected observations. Turn them into a direction argument.

For a more explicit end-to-end reasoning sequence, read references/idea-thinking-flow.md.

Creative-divergence protocol

Use deliberate ideation lenses before convergence when the route is not already obvious from durable evidence. The point is not uncontrolled brainstorming. The point is to widen the search just enough to avoid premature convergence onto the first implementable idea.

This divergence protocol does not replace the main workflow below. It sits inside the main workflow after minimum grounding already exists from memory reuse, initial literature sweep, baseline reconstruction, and limitation analysis. If strong durable evidence already narrows the route to one obvious serious option, you may abbreviate the full widening pass, but you must record why a broader divergence pass was unnecessary.

First classify the current entry frame:

• problem-first:
  • start from a concrete failure, bottleneck, or unmet need
  • confirm who suffers, how much it matters, and why the problem is still open
• solution-first:
  • start from a new capability, mechanism, or transfer idea
  • confirm at least two genuine problems it could solve and why this is not just a hammer looking for a nail

Then choose at least 2-4 ideation lenses that are actually relevant to the current bottleneck. Good default lenses include:

• abstraction ladder:
  • move up to a broader principle
  • move down to an extreme constrained case
  • move sideways to an adjacent task with the same structure
• tension or contradiction hunting:
  • identify tradeoffs such as performance vs efficiency, safety vs capability, or generality vs specialization
• why now / what changed:
  • ask whether new compute, tooling, open models, benchmarks, failures, or regulations make an old direction newly viable
• analogy transfer:
  • borrow a structural mechanism from a nearby or distant field only when the mapping is causal, not metaphorical
• constraint manipulation:
  • list hard, soft, and hidden constraints, then relax, tighten, or replace the soft or hidden ones
• negation or inversion:
  • negate a widely assumed design rule and check whether the resulting system is coherent
• composition / decomposition:
  • combine two complementary components or separate a monolithic method into the real bottleneck pieces
• adjacent possible:
  • focus on directions that became feasible only because recen

…(truncated)