Research

/research

End-to-end research orchestrator that composes all skills into a complete research workflow. Stage 0 (Bootstrap) + 5 Stages + 2 Human Gates, covering the full pipeline from empty wiki to paper submission. Zero-friction entry: if the wiki is empty, Bootstrap is triggered automatically (search + auto-ingest 5 papers); no need to run /init manually. Every Gate and Stage saves progress to wiki/outputs/pipeline-progress.md, supporting cross-session recovery.

Stage 3 is non-blocking: experiments are deployed and control returns immediately (--auto mode automatically sets up a CronCreate to monitor every 30 minutes). When all experiments finish, Stage 4 is triggered automatically. Use /exp-status at any time to check progress.

--auto mode skips manual confirmation (automatically selects the top-1 idea). --skip-paper runs the research without writing a paper.

Inputs

• direction: research direction description or path to a RESEARCH_BRIEF.md file
  • Text form: one-sentence description of the research direction (e.g. "sparse LoRA for edge devices")
  • File form: structured RESEARCH_BRIEF.md (containing domain, constraints, target venues)
• --auto (optional): fully automatic mode; Gate 1 auto-selects top-1 idea, Gate 2 auto-continues, Stage 3b auto-creates CronCreate
• --start-from <stage> (optional): resume execution from the specified stage
  • Valid values: stage1, stage2, stage3, stage3-collect, stage3-check, stage4, stage5
  • stage3-collect: skip deploy, go directly to Stage 3c (collect results from already-deployed experiments)
  • stage3-check: check experiment status only (equivalent to /exp-status --pipeline {slug}), do not continue execution
  • Requires wiki/outputs/pipeline-progress.md to exist
• --skip-paper (optional): run research only (Stages 1-4), skip paper writing (Stage 5), but still run /exp-eval (Stage 4)
• --venue (optional): target conference (ICLR / NeurIPS / ICML / ACL / CVPR), passed to /paper-plan

Outputs

• Wiki updates (delegated to sub-skills): ideas/, experiments/, claims/, outputs/, graph/
• wiki/outputs/pipeline-progress.md — pipeline progress snapshot (for recovery)
• wiki/outputs/PIPELINE_REPORT.md — full pipeline report
• paper/ directory (if not --skip-paper) — submittable paper
• wiki/log.md — log appended after each stage

Wiki Interaction

Reads

• wiki/graph/context_brief.md — global context (passed to sub-skills)
• wiki/graph/open_questions.md — knowledge gaps (passed to /ideate)
• wiki/ideas/*.md — Gate 1 selection, Stage 4 verdict
• wiki/experiments/*.md — Stage 3-4 status checks
• wiki/claims/*.md — Stage 4 verdict, Stage 5 paper planning
• wiki/outputs/pipeline-progress.md — --start-from state recovery
• wiki/papers/*.md — Stage 5 paper writing context

Writes

• wiki/outputs/pipeline-progress.md — save progress at each Gate (wiki entity writes are delegated to sub-skills)
• wiki/outputs/PIPELINE_REPORT.md — final report
• wiki/log.md — append log entries
• All other wiki entity writes are delegated to sub-skills (do not directly write to ideas/experiments/claims/)

Graph edges created

• None directly — all graph edges are delegated to sub-skills (/ideate, /exp-design, /exp-eval each create their own edges)

Workflow

Precondition:

1. Confirm working directory is the wiki project root (containing wiki/, raw/, tools/)
2. If --start-from is specified, read wiki/outputs/pipeline-progress.md to restore state

Step 0: Initialize

1. Parse input:

   • If file path: read RESEARCH_BRIEF.md, extract direction, domain, constraints, target_venue
   • If text: use as direction; leave domain/constraints blank
   • Generate slug: python3 tools/research_wiki.py slug "{direction}"

2. Auto-recovery detection (when --start-from is not specified):

   • If wiki/outputs/pipeline-progress.md exists and status == running:
     • Read direction, current_stage, started, slug
     • Use AskUserQuestion to prompt the user:

       Unfinished pipeline detected:
       Direction: {direction}
       Current stage: {current_stage}
       Started: {started}
       
       [1] Resume from {current_stage} (recommended)
       [2] Start a new pipeline (will overwrite old progress)
       [3] View experiment status first (/exp-status --pipeline {slug})
       

     • If --auto or user selects [1]: auto-set --start-from {current_stage}, continue execution
     • If user selects [2]: continue creating new pipeline (overwrite old progress file)
     • If user selects [3]: call /exp-status --pipeline {slug} then exit without continuing

3. Check recovery (when --start-from is specified):

   • If wiki/outputs/pipeline-progress.md exists:
     • Read progress file, restore idea_slug, experiment_slugs, stage3a_deployed, claim_slugs, monitoring_cron_id
     • Jump to specified stage
   • If progress file does not exist: report error and exit; prompt user to run the full pipeline first
   • --start-from stage3-check: equivalent to calling /exp-status --pipeline {slug}; display status then exit
   • --start-from stage3-collect: skip Stage 3a+3b; go directly to Stage 3c (collect already-deployed experiments)

4. Create progress file wiki/outputs/pipeline-progress.md:

   ---
   slug: "{pipeline-slug}"
   direction: "{research direction}"
   status: running
   current_stage: stage1
   started: YYYY-MM-DD
   mode: auto|interactive
   skip_paper: true|false
   venue: "{venue}"
   idea_slug: ""
   experiment_slugs: []
   stage3a_deployed: []
   claim_slugs: []
   iteration_count: 0
   ---
   ## Stage Log
   - Stage 0 (Bootstrap): skipped
   - Stage 1: pending
   - Gate 1: pending
   - Stage 2: pending
   - Stage 3a (Deploy): pending
   - Stage 3b (Await): pending
   - Stage 3c (Collect): pending
   - Stage 4: pending
   - Gate 2: pending
   - Stage 5: pending
   

5. Append log:

   python3 tools/research_wiki.py log wiki/ \
     "research | started | direction: {direction} | mode: {auto|interactive}"
   

6. Snapshot wiki state (for Growth Report in Step Final):

   python3 tools/research_wiki.py maturity wiki/ --json
   

   Save returned JSON to memory variable maturity_before.

Stage 0: Bootstrap (triggered automatically when wiki is empty)

Trigger condition: run python3 tools/research_wiki.py maturity wiki/ --json. If level == "cold" and papers < 3: enter Bootstrap automatically. Otherwise skip and proceed to Stage 1.

1. Initialize wiki structure (if not yet initialized):

   python3 tools/research_wiki.py init wiki/
   

2. Search for relevant papers (use Agent tool with 3 parallel searches):

   • DeepXiv: python3 tools/fetch_deepxiv.py search "{direction}" --mode hybrid --limit 20
   • Semantic Scholar: python3 tools/fetch_s2.py search "{direction}" --limit 20
   • arXiv: python3 tools/fetch_arxiv.py (using direction keywords)
   • If DeepXiv is unavailable: skip; use only S2 + arXiv

3. Merge, rank, and select top 5:

   • Deduplicate by arxiv_id
   • Ranking priority: DeepXiv relevance score > S2 citation count > recency
   • Select top 5 (5 = minimum threshold for cold→warm)

4. Auto-ingest each paper:

   Skill: ingest
   Args: "{arxiv_url_or_path}"
   

   Output progress after each ingest: [{i}/5] Ingested: {paper_title}

5. Rebuild derived data:

   python3 tools/research_wiki.py rebuild-context-brief wiki/
   python3 tools/research_wiki.py rebuild-open-questions wiki/
   

6. Bootstrap report:

   python3 tools/research_wiki.py maturity wiki/ --json
   

   Output to terminal:

   Bootstrap complete:
   Papers: {N} | Claims: {M} | Concepts: {K} | Edges: {E}
   Maturity: cold → {new_level}
   Proceeding to Stage 1: Idea Discovery...
   

7. Log + update progress:

   python3 tools/research_wiki.py log wiki/ \
     "research | stage0-bootstrap | auto-ingested {N} papers | maturity: {level}"
   python3 tools/research_wiki.py set-meta \
     wiki/outputs/pipeline-progress.md current_stage stage1
   

Stage 1: Idea Discovery

Call /ideate:

Skill: ideate
Args: "{direction}" --domain {domain}

After completion:

1. Read the generated ideas, sorted by priority
2. Update pipeline-progress: Stage 1 → completed, record generated idea slugs
3. Append log

Gate 1: Select Idea

If --auto mode:

• Automatically select the highest-priority (top-1) idea
• Output selection result to terminal without waiting for confirmation

If interactive mode:

• List all generated ideas (slug, title, priority, novelty score)
• Use AskUserQuestion to prompt user to select one idea (or enter "stop" to halt)
• If user selects stop: save progress, terminate pipeline

Save progress:

• Update pipeline-progress: Gate 1 → passed, record idea_slug
• Update selected idea status: proposed → in_progress

Stage 2: Experiment Design

Call /exp-design:

Skill: exp-design
Args: "{idea_slug}" --review

After completion:

1. Read generated experiment slugs (pages in wiki/experiments/ where linked_idea == idea_slug)
2. Update pipeline-progress: Stage 2 → completed, record experiment_slugs

Stage 3: Experiment Execution (non-blocking)

Stage 3 is divided into three sub-stages, allowing experiments to run asynchronously in the background without blocking the session.

Stage 3a: Deploy All

Deploy each experiment in run order (baseline → validation → ablation → robustness) by calling /exp-run {experiment_slug} (default deploy mode, Phase 1+2):

Skill: exp-run
Args: "{experiment_slug}"

(Default deploy mode, Phase 1+2: returns immediately after deployment, does not wait for experiment to finish)

After each deployment:

• Record deployment result (success/failure) in memory
• If deploy fails: record to pipeline-progress with a warning (baseline deploy failure gets a stronger warning), but continue deploying remaining experiments (do not abort)

After all deployments complete, update pipeline-progress.md:

python3 tools/research_wiki.py set-meta \
  wiki/outputs/pipeline-progress.md current_stage stage3-await
python3 tools/research_wiki.py set-meta \
  wiki/outputs/pipeline-progress.md stage3a_deployed \
  "[{experiment_slug_1}, {experiment_slug_2}, ...]"

Append log:

python3 tools/research_wiki.py log wiki/ \
  "research | stage3a | deployed {N} experiments | pipeline: {slug}"

Stage 3b: Await (non-blocking)

After all experiments are deployed, compute ETA, save progress, and end the current session.

1. Update pipeline-progress:

   python3 tools/research_wiki.py set-meta \
     wiki/outputs/pipeline-progress.md current_stage stage3-await
   

2. Compute estimated completion time for each experiment: For each deployed experiment, read started and estimated_hours from frontmatter:
   • eta = started + estimated_hours
   • recommended_return = max(all etas) + 30-minute buffer, rounded up to nearest hour or half-hour
3. Append log:

   python3 tools/research_wiki.py log wiki/ \
     "research | stage3b | awaiting {N} experiments | latest eta: {YYYY-MM-DD HH:MM} | pipeline: {slug}"
   

4. Output instructions then end current session:

   Stage 3a complete: {N} experiments all deployed:
   
   Experiment                      Environment     Est. Duration   Est. Completion
   ──────────────────────────────  ──────────────  ─────────────   ───────────────
   exp-foo-baseline                local           ~8h             Tomorrow 09:30
   exp-foo-validation              remote (gpu1)   ~6h             Today 23:00
   exp-foo-ablation                local           ~4h             Today 21:00
   
   Latest completion: Tomorrow 09:30 (exp-foo-baseline)
   Recommended time to return: Tomorrow 10:00+
   
     /exp-status                              ← confirm all experiments complete
     /research --start-from stage3-collect    ← collect results and continue
   
   Progress saved to wiki/outputs/pipeline-progress.md; current session can be closed.
   

Stage 3c: Collect (triggered after experiments complete)

Trigger: user manually runs /research --start-from stage3-collect

For each deployed experiment (read from stage3a_deployed list):

Skill: exp-run
Args: "{experiment_slug} --collect"

(Collect mode, Phase 3+4: check completion status and collect results)

Decision after each collect:

• If outcome == failed and this is the baseline experiment → terminate pipeline, report baseline cannot be reproduced
• If outcome == failed and this is a validation experiment → record failure, continue collecting remaining experiments, proceed to Stage 4 evaluation
• If outcome == inconclusive → record and continue

After all collects complete:

• Update pipeline-progress: Stage 3 → completed

  python3 tools/research_wiki.py set-meta \
    wiki/outputs/pipeline-progress.md current_stage stage4
  

• Append log:

  python3 tools/research_wiki.py log wiki/ \
    "research | stage3c | collected {N} experiments | pipeline: {slug}"
  

• Proceed to Stage 4

Stage 4: Verdict & Iteration

Call /exp-eval for each completed experiment:

Skill: exp-eval
Args: "{experiment_slug}" --auto

Evaluate whether claims are sufficient:

1. Read the latest status of all target claims
2. Determine whether iteration is needed:
   • Claims sufficient (primary claim confidence >= 0.7 and status is supported or weakly_supported) → proceed to Gate 2
   • Claims insufficient (confidence < 0.4 or status is challenged) → enter iteration

Iteration path (when claims are insufficient, up to 1 retry):

1. Analyze the cause of failure
2. Call /refine to improve the experiment plan:

   Skill: refine
   Args: "{experiment_plan_slug}" --max-rounds 2 --focus evidence
   

3. Re-run Stage 3 → Stage 4 for new/modified experiments
4. Maximum 2 iterations (prevents infinite loops); each stage has at most 1 auto-retry

After completion:

• Update pipeline-progress: Stage 4 → completed, record claim_slugs

Gate 2: Confirm Paper Ready

If --skip-paper: skip Gate 2 and Stage 5, generate final report directly

If --auto mode: automatically continue, enter Stage 5

If interactive mode:

• Display claim status summary:

  Claim: {slug} | Status: {status} | Confidence: {confidence}
  Evidence: {count} sources ({strong}/{moderate}/{weak})
  

• Use AskUserQuestion to prompt user: ready for paper / need more experiments / stop here
• If "need more experiments": return to Stage 2 for replanning
• If "stop here": save progress, generate final report (without paper)

Save progress:

• Update pipeline-progress: Gate 2 → passed

Stage 5: Paper Writing

Call sub-skills in sequence: /paper-plan → /paper-draft → /refine → /paper-compile

5a. Call /paper-plan:

Skill: paper-plan
Args: "{claim_slugs}" --venue {venue}

5b. Call /paper-draft:

Skill: paper-draft
Args: "wiki/outputs/PAPER_PLAN.md" --review

5c. Call /refine on paper:

Skill: refine
Args: "paper/main.tex" --max-rounds 3 --target-score 8 --focus writing

5d. Call /paper-compile:

Skill: paper-compile
Args: "paper/"

After completion:

• Update pipeline-progress: Stage 5 → completed, status: completed

Step Final: Pipeline Report

Generate wiki/outputs/PIPELINE_REPORT.md:

# Research Pipeline Report

## Stage Summary
| Stage | Status | Duration |
|-------|--------|----------|
| Stage 0: Bootstrap | completed/skipped | ... |
| Stage 1: Idea Discovery | completed | ... |
| Gate 1: Idea Selection | passed | ... |
| Stage 2: Experiment Design | completed | ... |
| Stage 3a: Deploy Experiments | completed | ... |
| Stage 3b: Await (async) | completed | ... |
| Stage 3c: Collect Results | completed | ... |
| Stage 4: Verdict | completed | ... |
| Gate 2: Paper Ready | passed | ... |
| Stage 5: Paper Writing | completed | ... |

## Selected Idea
- **Idea**: [[{idea_slug}]] — {idea title}
- **Priority**: {N}
- **Novelty score**: {score}

## Claims Trail
| Claim | Initial Status | Final Status | Confidence (proposed → supported) |
|-------|---------------|-------------|-----------------------------------|
| [[{slug}]] | proposed | supported | 0.3 → 0.8 |

## Experiment Results
| Experiment | Outcome | Key Result |
|-----------|---------|------------|
| [[{slug}]] | succeeded | {result} |

## Iteration History
- Total iterations: {N}
- Reason for iteration: {claims insufficient / ...}

## Deliverables
- Ideas: +{N} created
- Experiments: +{N} created, {N} completed
- Claims: {N} updated
- Graph edges: +{N}
- Paper: paper/main.pdf (if applicable)

## Wiki Growth (pipeline total)
| Metric | Before | After | Delta |
|--------|--------|-------|-------|
| Papers | {N} | {N} | +{N} |
| Claims | {N} | {N} | +{N} |
| Ideas | {N} | {N} | +{N} |
| Experiments | {N} | {N} | +{N} |
| Edges | {N} | {N} | +{N} |
| Maturity | {level} | {level} | {status} |
| Coverage | {%} | {%} | +{%} |
(Data from comparing `maturity_before` from Step 0 against a fresh call to `maturity --json` here. Only rows with delta != 0 are shown.)

## Next Steps
- {recommendations based on remaining gaps or unresolved issues}

Append log:

python3 tools/research_wiki.py log wiki/ \
  "research | completed | idea: {slug} | claims: {N} updated | paper: {yes/no}"

Update pipeline-progress: status: completed

Constraints

• Orchestrator does not directly modify wiki entities or embed sub-skill logic: all wiki modifications are delegated to sub-skills; the pipeline only coordinates by calling them via the Skill tool
• Gates and Stages must save progress: every Gate and Stage must save pipeline-progress.md when completed or entering await
• Stage 3a deploy failures do not abort: record a warning and continue deploying; do not terminate early (baseline collect failure is what triggers termination)
• Baseline collect failure terminates: in Stage 3c, if baseline outcome == failed, terminate the pipeline
• Stage 3b ends the session: after Stage 3b completes, the current session ends; do not continue waiting for experiments
• Maximum 2 iterations: Stage 4 iterates at most 2 times to prevent infinite loops
• --auto does not skip computation: auto mode skips human confirmation but skips no computation steps
• --skip-paper still runs Stage 4 /exp-eval: claim updates must be completed even when not writing a paper
• Pass sub-skill parameters through: correctly pass domain, --venue, and other parameters to sub-skills
• Log every Stage: append a log.md audit entry after each Stage completes
• Do not re-run completed stages: --start-from skips already-completed stages
• Progress file at wiki/outputs/pipeline-progress.md: consistent location for easy discovery and recovery
• Auto-recovery first: if no --start-from is given and an unfinished pipeline exists, default to prompting the user to resume rather than starting fresh

Error Handling

• pipeline-progress missing but --start-from specified: report error; prompt user to run the full pipeline first
• pipeline-progress corrupted or malformed: attempt to infer progress from current wiki state (read ideas/experiments/claims statuses), recover to the nearest Gate
• Sub-skill call fails: record error to pipeline-progress, report the failed stage, suggest --start-from to resume
• All ideas generation fails: terminate pipeline; suggest the user adjust the research direction
• All experiment deploys fail: terminate pipeline (Stage 3a); generate failure report; suggest checking GPU/SSH configuration
• Stage 3c baseline collect fails: terminate pipeline; report baseline cannot be reproduced; suggest re-running /exp-design
• All experiment collects fail (non-baseline): proceed to Stage 4 evaluation (treat failures as evidence)
• Gate user selects stop: save progress to pipeline-progress; generate partial report
• RESEARCH_BRIEF.md malformed: fall back to plain-text direction; ignore structured fields
• Wiki empty (no papers/concepts): auto-trigger Stage 0 Bootstrap (search + auto-ingest 5 papers)
• Claims still insufficient after iteration: annotate report with "claims insufficient after max iterations"; let user decide whether to continue
• User selects view status (auto-recovery detection [3]): call /exp-status --pipeline {slug} then exit without starting a new pipeline

Dependencies

Skills（via Skill tool）

• /ingest — Stage 0 Bootstrap auto-ingest
• /ideate — Stage 1 idea discovery
• /exp-design — Stage 2 experiment design
• /exp-run — Stage 3a (deploy mode) and Stage 3c (--collect mode)
• /exp-status — user manually checks experiment progress; --auto-advance can automatically trigger Stage 4 when all complete
• /exp-eval — Stage 4 verdict
• /refine — Stage 4 iteration + Stage 5 paper improvement
• /paper-plan — Stage 5 paper planning
• /paper-draft — Stage 5 paper writing
• /paper-compile — Stage 5 paper compilation

Tools（via Bash）

• python3 tools/research_wiki.py slug "{title}" — generate pipeline slug
• python3 tools/research_wiki.py set-meta <path> <field> <value> — update pipeline-progress fields
• python3 tools/research_wiki.py log wiki/ "<message>" — append log entry
• python3 tools/research_wiki.py maturity wiki/ --json — check wiki maturity (Stage 0 trigger + Growth Report)
• python3 tools/research_wiki.py init wiki/ — initialize wiki structure (Stage 0)
• python3 tools/fetch_deepxiv.py search "{query}" --mode hybrid --limit 20 — DeepXiv semantic search (Stage 0)
• python3 tools/fetch_s2.py search "{query}" --limit 20 — Semantic Scholar search (Stage 0)
• python3 tools/fetch_arxiv.py — arXiv RSS search (Stage 0)

MCP Servers

• None directly — all Review LLM interactions are used indirectly via sub-skills

Claude Code Native

• Read — read pipeline-progress, wiki pages, RESEARCH_BRIEF
• Write — write pipeline-progress, PIPELINE_REPORT
• Glob — find experiments, ideas, claims
• Skill — call sub-skills (core capability)
• AskUserQuestion — user interaction at Gates and auto-recovery detection