Autoresearch

Autoresearch: Autonomous Experiment Loop

Adapted from Feynman's autoresearch methodology for Claude Code + RuVector memory.

The six-step loop below (Bash/Read/Write/Edit/Grep/Glob only) runs identically on Codex / GPT-6 Astra. Claude Code only: running several experiment branches in parallel would need the Agent/fork tool, which this skill does not use by default; on Codex / GPT-6 Astra, run branches sequentially instead.

Step 1: Gather Context

If resuming, search RuVector memory first:

mcp__claude-flow__memory_search({query: "[topic] autoresearch experiments", namespace: "patterns", limit: 10})

If starting fresh, collect from the user:

• Metric: What to optimize (latency, throughput, bundle size, GPU kernel time, accuracy, etc.)
• Benchmark command: The shell command that produces the metric
• Direction: Lower or higher is better
• Files in scope: Which source files can be modified
• Max iterations: Default 20
• Environment: Local working directory (default), git worktree branch, or Docker

Step 2: Confirm Plan

Present the plan and get explicit approval before starting:

Optimization target: [metric] ([direction])
Benchmark command:   [command]
Files in scope:      [files]
Environment:         [environment]
Max iterations:      [N]

Step 3: Baseline

Run the benchmark command once to establish baseline:

# Record baseline
time [benchmark_command] 2>&1 | tee /tmp/autoresearch-baseline.txt

Log baseline to experiment journal:

echo "$(date -Iseconds) | baseline | [metric_value] | [unit] | initial state" >> outputs/autoresearch.jsonl

Step 4: Experiment Loop

For each iteration (1 to max_iterations):

1. Hypothesize: Based on prior results, identify the most promising change
2. Implement: Edit the files in scope (use Edit tool, keep changes minimal and reversible)
3. Measure: Run the benchmark command
4. Log: Record the result

   echo "$(date -Iseconds) | iter_[N] | [metric_value] | [unit] | [description of change]" >> outputs/autoresearch.jsonl
   

5. Decide:
   • If metric improved: KEEP the change, commit with git commit -m "autoresearch: [change] ([metric improvement])"
   • If metric regressed or unchanged: REVERT with git checkout -- [files]
6. Learn: Store successful patterns in RuVector:

   mcp__claude-flow__memory_store({namespace: "patterns", key: "autoresearch-[slug]-iter[N]", value: "[what worked/failed]"})
   

Step 5: Convergence

Stop when:

• Max iterations reached
• 3 consecutive iterations with < 1% improvement
• User interrupts

Step 6: Report

Write final report to outputs/autoresearch-[slug]-report.md:

# Autoresearch Report: [topic]

## Result
- Baseline: [value] [unit]
- Final: [value] [unit]
- Improvement: [percentage]
- Iterations: [N] ([kept] kept, [reverted] reverted)

## What Worked
1. [change] → [improvement]

## What Didn't Work
1. [change] → [regression or no effect]

## Experiment Log
[table from autoresearch.jsonl]

Store final result in memory:

mcp__claude-flow__memory_store({namespace: "patterns", key: "autoresearch-[slug]-final", value: "[summary of what worked, final metric, key learnings]"})

Subcommands

• /autoresearch <topic> — start or resume the loop
• /autoresearch status — show current iteration and best result
• /autoresearch stop — stop the loop, keep data

Integration with GPU Pipeline

When optimizing GPU kernels, use:

• nvidia-smi for GPU utilization and memory monitoring
• CUDA event timing via the benchmark command
• cargo bench or custom benchmark binaries
• Kernel launch parameter sweeps (block size, grid size)