How do I install the Setup skill?

Run npx skillmds@latest add sinhoneyy/setup in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Setup safe to use?

SkillMD's automated safety review verdict for this skill is PASS. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. Capability flags: docs only. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Setup?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Setup free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

sinhoneyy (@sinhoneyy) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Setup by sinhoneyy — Agent Skill

Name: Setup
Rating: 4.666666666666667 (3 reviews)
Author: sinhoneyy

Setup

Set up a new autoresearch experiment interactively. Collects domain, target file, eval command, metric, direction, and evaluator. Use when the user runs /ar:setup or asks to start optimizing a file with the autoresearch loop.

sinhoneyy Updated Aug 19, 2026 11 repo stars

File contents

/ar:setup — Create New Experiment

Set up a new autoresearch experiment with all required configuration.

Usage

/ar:setup                                    # Interactive mode
/ar:setup engineering api-speed src/api.py "pytest bench.py" p50_ms lower
/ar:setup --list                             # Show existing experiments
/ar:setup --list-evaluators                  # Show available evaluators

What It Does

If arguments provided

Pass them directly to the setup script:

python {skill_path}/scripts/setup_experiment.py \
  --domain {domain} --name {name} \
  --target {target} --eval "{eval_cmd}" \
  --metric {metric} --direction {direction} \
  [--evaluator {evaluator}] [--scope {scope}]

If no arguments (interactive mode)

Collect each parameter one at a time:

Domain — Ask: "What domain? (engineering, marketing, content, prompts, custom)"
Name — Ask: "Experiment name? (e.g., api-speed, blog-titles)"
Target file — Ask: "Which file to optimize?" Verify it exists.
Eval command — Ask: "How to measure it? (e.g., pytest bench.py, python evaluate.py)"
Metric — Ask: "What metric does the eval output? (e.g., p50_ms, ctr_score)"
Direction — Ask: "Is lower or higher better?"
Evaluator (optional) — Show built-in evaluators. Ask: "Use a built-in evaluator, or your own?"
Scope — Ask: "Store in project (.autoresearch/) or user (~/.autoresearch/)?"

Then run setup_experiment.py with the collected parameters.

Listing

# Show existing experiments
python {skill_path}/scripts/setup_experiment.py --list

# Show available evaluators
python {skill_path}/scripts/setup_experiment.py --list-evaluators

Built-in Evaluators

Name	Metric	Use Case
`benchmark_speed`	`p50_ms` (lower)	Function/API execution time
`benchmark_size`	`size_bytes` (lower)	File, bundle, Docker image size
`test_pass_rate`	`pass_rate` (higher)	Test suite pass percentage
`build_speed`	`build_seconds` (lower)	Build/compile/Docker build time
`memory_usage`	`peak_mb` (lower)	Peak memory during execution
`llm_judge_content`	`ctr_score` (higher)	Headlines, titles, descriptions
`llm_judge_prompt`	`quality_score` (higher)	System prompts, agent instructions
`llm_judge_copy`	`engagement_score` (higher)	Social posts, ad copy, emails

After Setup

Report to the user:

Experiment path and branch name
Whether the eval command worked and the baseline metric
Suggest: "Run /ar:run {domain}/{name} to start iterating, or /ar:loop {domain}/{name} for autonomous mode."

---name: setupdescription: Set up a new autoresearch experiment interactively. Collects domain, target file, eval command, metric, direction, and evaluator. Use when the user runs /ar:setup or asks to start optimizing a file with the autoresearch loop.---# /ar:setup — Create New ExperimentSet up a new autoresearch experiment with all required configuration.## Usage```/ar:setup # Interactive mode/ar:setup engineering api-speed src/api.py "pytest bench.py" p50_ms lower/ar:setup --list # Show existing experiments/ar:setup --list-evaluators # Show available evaluators```## What It Does### If arguments providedPass them directly to the setup script:```bashpython {skill_path}/scripts/setup_experiment.py \ --domain {domain} --name {name} \ --target {target} --eval "{eval_cmd}" \ --metric {metric} --direction {direction} \ [--evaluator {evaluator}] [--scope {scope}]```### If no arguments (interactive mode)Collect each parameter one at a time:1. **Domain** — Ask: "What domain? (engineering, marketing, content, prompts, custom)"2. **Name** — Ask: "Experiment name? (e.g., api-speed, blog-titles)"3. **Target file** — Ask: "Which file to optimize?" Verify it exists.4. **Eval command** — Ask: "How to measure it? (e.g., pytest bench.py, python evaluate.py)"5. **Metric** — Ask: "What metric does the eval output? (e.g., p50_ms, ctr_score)"6. **Direction** — Ask: "Is lower or higher better?"7. **Evaluator** (optional) — Show built-in evaluators. Ask: "Use a built-in evaluator, or your own?"8. **Scope** — Ask: "Store in project (.autoresearch/) or user (~/.autoresearch/)?"Then run `setup_experiment.py` with the collected parameters.### Listing```bash# Show existing experimentspython {skill_path}/scripts/setup_experiment.py --list# Show available evaluatorspython {skill_path}/scripts/setup_experiment.py --list-evaluators```## Built-in Evaluators| Name | Metric | Use Case ||------|--------|----------|| `benchmark_speed` | `p50_ms` (lower) | Function/API execution time || `benchmark_size` | `size_bytes` (lower) | File, bundle, Docker image size || `test_pass_rate` | `pass_rate` (higher) | Test suite pass percentage || `build_speed` | `build_seconds` (lower) | Build/compile/Docker build time || `memory_usage` | `peak_mb` (lower) | Peak memory during execution || `llm_judge_content` | `ctr_score` (higher) | Headlines, titles, descriptions || `llm_judge_prompt` | `quality_score` (higher) | System prompts, agent instructions || `llm_judge_copy` | `engagement_score` (higher) | Social posts, ad copy, emails |## After SetupReport to the user:- Experiment path and branch name- Whether the eval command worked and the baseline metric- Suggest: "Run `/ar:run {domain}/{name}` to start iterating, or `/ar:loop {domain}/{name}` for autonomous mode."

Frequently asked questions