# Benchmark

> Use when you need measured performance evidence by running a repeatable command on the current branch and a baseline ref

- Skill: `josephneumann/benchmark` (Agent Skill)
- Install (CLI): `npx skillmds@latest add josephneumann/benchmark`
- Raw SKILL.md: https://api.skillmd.com/api/skills/josephneumann/benchmark/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: josephneumann (https://skillmd.com/u/josephneumann)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/josephneumann/benchmark

---


# /benchmark $ARGUMENTS

Run a repeatable benchmark command on the current branch and compare it with a baseline ref. This skill is for measurement, not speculation.

If the user wants architectural performance review without running code, use `/multi-review` with `performance-oracle` instead.

## Arguments

```text
/benchmark "<command>" [--baseline <ref>] [--iterations N]
```

Defaults:
- `--baseline origin/main`
- `--iterations 5`
- warmup runs: `1`

Reject the request if no command is provided.

## Safety Rules

- The benchmark command must be repeatable
- Do not use commands that intentionally mutate repo-tracked files
- Build artifacts and caches are acceptable if they are not tracked
- If the command dirties the worktree, stop and report it

## Phase 1: Pre-Flight

1. Record current branch and `git status --short`
2. Verify the worktree is clean enough to benchmark safely
3. Resolve the baseline ref
4. Determine whether the benchmark command depends on repo-local dependencies, generated assets, or setup steps
5. Create a temporary detached worktree for the baseline
6. Establish equivalent runtime environments for current and baseline before timing anything

Use a temporary directory under `/tmp` or equivalent. Never benchmark the baseline by checking out over the user's current branch.

Environment setup rules:
- Read local sources of truth first: `CLAUDE.md`, `README.md`, and project manifests such as `package.json`, `pyproject.toml`, `Cargo.toml`, or `Makefile`
- If the benchmark command depends on installed dependencies or generated artifacts, run the same setup flow in both environments before measuring
- Prefer the project's existing setup command if one is documented, for example `pnpm install --frozen-lockfile`, `uv sync`, `cargo fetch`, or `make setup`
- Do not count setup time as benchmark time
- If you cannot establish equivalent environments on both refs, stop and report that the benchmark would be untrustworthy

## Phase 2: Measurement Protocol

For both current branch and baseline:

1. Run 1 warmup execution
2. Run `N` measured executions
3. Capture wall-clock duration for every measured run
4. If the command prints benchmark metrics already, include them in the notes, but wall-clock time is the required common metric

Prefer a consistent timing mechanism for every run. Keep environment conditions as similar as possible across both refs.

## Phase 3: Summarize Results

Compute for current and baseline:
- all measured run times
- median
- min
- max
- percentage delta of median vs baseline

If run-to-run variance is high, say so. Do not overclaim a tiny difference hidden by noise.

## Required Output

```markdown
## Benchmark Summary

Command: ...
Baseline: ...
Iterations: ...

### Current Branch
Runs: [...]
Median: ...
Min/Max: ...

### Baseline
Runs: [...]
Median: ...
Min/Max: ...

### Delta
Current vs baseline median: ...%

### Notes
- command-native metrics if any
- variance caveats
- anything that may have skewed results
```

## Cleanup

Always remove the temporary baseline worktree before exiting, even on failure.

## Failure Handling

- If the benchmark command fails on either ref, report the failing ref and stop
- If the baseline ref cannot be resolved, stop
- If cleanup fails, report it explicitly

## Interpretation Rules

- Measured improvement beats theoretical reasoning
- High variance weakens confidence
- A benchmark proves only the command that was run, not overall product performance