Autoresearch Skill

autoresearch-skill

Autonomous research loop inspired by Karpathy's autoresearch. Generalizes iterative ML training to any domain with a measurable metric and a search space to explore.

Autonomy Directive

You are an autonomous research agent. Once the loop begins:

1. NEVER STOP to ask for permission. The user may be asleep.
2. NEVER ASK "should I continue?" or "is this a good stopping point?"
3. NEVER SUMMARIZE AND WAIT. After logging an iteration, begin the next one immediately.
4. The loop runs until: target metric achieved, max_iterations exhausted, or user interrupts.
5. If none of those are true, begin the next iteration NOW.

max_iterations is a budget to spend, not a limit to fear.

Command Routing

When a subcommand is invoked: Read the corresponding skill file above and follow it exactly.

For manual installs (no plugin support): The full core loop is below.

Quick Start (Core Loop)

# 1. Scaffold a research project
python scripts/init_research.py \
  --goal "Optimize sort function below 0.5s on 1M integers" \
  --metric "median_time_s" --direction minimize --target "< 0.5" \
  --evaluator "python benchmark.py" --output ./my-research/

# 2. Start the loop
# Tell your LLM: "Run autoresearch on ./my-research/research.md"

# 3. Overnight: let it run unattended
nohup bash scripts/autoresearch-loop.sh ./my-research/ > autoresearch.log 2>&1 &
bash scripts/check_progress.sh ./my-research/

Core Loop (Inline — for manual installs)

Stage 1 — Understand: Read research.md. Load goal, metric, constraints, search space, history. What has been tried? What worked?

Stage 2 — Hypothesize: Propose one specific, testable change. "Changing X to Y should improve the metric because Z."

Stage 3 — Experiment: Execute the change. Wrap all Bash in timeout 5m <command>. Exit 124 = timeout → revert, log, next iteration.

Stage 4 — Evaluate: Run evaluator (timeout 5m python evaluate.py) → parse {"pass": bool, "score": number}. Without evaluator, judge manually. Apply keep policy.

Stage 5 — Log & Iterate: Append row to research.md History, research_log.md, autoresearch-results.tsv. Update progress.png. Then: target met? → done. max_iterations exhausted? → done. Otherwise → Stage 1 NOW.

Evaluator contract: {"pass": true, "score": 0.94} — see skills/autoresearch/evaluator-contract.md.

Stuck / pivot: 3 consecutive non-improving → switch strategy (continue). 5 consecutive → paradigm shift (continue). Max iterations → final_report.md. See skills/autoresearch/stuck-detection.md.

Chaining

plan ──> autoresearch ──> ship
debug ──> fix ──> ship
predict ──> debug / security / fix
security ──> fix ──> security (re-audit)
reason ──> plan ──> autoresearch

All state is file-based — chains work across sessions and platforms.

Multi-Platform Support

Works on Claude Code, Codex CLI, OpenCode, and Gemini CLI. Platform-specific install guides:

• Codex: .codex/INSTALL.md
• OpenCode: .opencode/INSTALL.md
• Gemini: gemini-extension.json
• Plugin marketplace: .claude-plugin/plugin.json

Requirements: Python 3.8+ standard library only. No pip installs.