How do I install the Bench skill?

Run npx skillmds@latest add nagisanzenin/bench in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Bench safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Bench?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Bench free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

nagisanzenin (@nagisanzenin) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Bench by nagisanzenin — Agent Skill

Name: Bench
Author: nagisanzenin

Bench

Run the strictlybetter meta-benchmark: the walled loop against a naive loop on fixture repos, reporting false-accept rate and cost per accepted improvement. Use when the user asks to benchmark, validate, or compare strictlybetter itself (not the target project).

nagisanzenin 9838229 3.2 KB Updated Sep 17, 2026

File contents

/strictlybetter:bench — the meta-benchmark

The loop's own claim is testable (docs/10 §10.7). The runner is a separate program; this skill only drives it.

# Resolve the engine. RUN THIS BLOCK VERBATIM — do not substitute a path you guessed.
# Order: ZCode's plugin root (ZCode also exports the legacy CLAUDE_PLUGIN_ROOT, so its own
# var is checked first), Claude Code, Codex, an explicit checkout (SB_ROOT), the Claude Code
# plugin cache (newest version), and LAST the working tree's git toplevel (a contributor
# running inside the strictlybetter checkout). First one that holds scripts/sb.py wins.
SB_PY=""
SB_CACHE="$(find "$HOME/.claude/plugins/cache" -maxdepth 5 -path '*/strictlybetter/*/scripts/sb.py' 2>/dev/null | sort | tail -n 1)"
for d in "${ZCODE_PLUGIN_ROOT:-}" "${CLAUDE_PLUGIN_ROOT:-}" "${CODEX_PLUGIN_ROOT:-}" "${SB_ROOT:-}" \
         "${SB_CACHE%/scripts/sb.py}" "$(git rev-parse --show-toplevel 2>/dev/null)"; do
  [ -n "$d" ] && [ -f "$d/scripts/sb.py" ] && SB_PY="$d/scripts/sb.py" && break
done
if [ -z "$SB_PY" ]; then
  echo "strictlybetter: engine not found — set SB_ROOT to your strictlybetter checkout" >&2
  return 2 2>/dev/null || exit 2   # FAIL CLOSED: proceeding would run `python3 ""`
fi
SB_ROOT="${SB_PY%/scripts/sb.py}"
SB_REPO="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
sb() { python3 "$SB_PY" --repo "$SB_REPO" "$@"; }   # a FUNCTION, not a string: zsh does not
SB=sb                                                 # word-split "$SB"; both shells run `sb`
$SB --version | grep -q '^sb 1\.' || { echo "strictlybetter: engine did not answer (expected sb 1.x)" >&2; return 2 2>/dev/null || exit 2; }   # major version only; exact patch level is not asserted

0 · Re-anchor (never trust conversational memory)

$SB status --json                                             # the truth on disk
$SB status --json | grep -q '"campaign": null' || $SB next    # the cold-start brief, when a campaign exists

1 · Entry point

ls "$SB_ROOT/bench/" 2>/dev/null || echo "bench runner not installed in this checkout"
python3 "$SB_ROOT/bench/run_bench.py" --help

Follow the usage the runner prints; pass the user's arguments through unchanged. Do not invent flags. Runs are long (each condition drives full campaigns on the fixture repos under tests/fixtures/); say so before starting and prefer running it in the background.

2 · Results

Results land in $SB_ROOT/bench/results/ (one directory or file per run, written by the runner, never hand-edited). Print the path of the newest result and quote its summary table verbatim. The two numbers that matter are the false-accept rate on a fresh holdout for each condition (all walls vs naive) and cost per accepted improvement. If the walls do not reduce the false-accept rate at acceptable cost, say so plainly; the thesis is falsifiable and the docs promise to report it either way.

---name: benchdescription: Run the strictlybetter meta-benchmark: the walled loop against a naive loop on fixture repos, reporting false-accept rate and cost per accepted improvement. Use when the user asks to benchmark, validate, or compare strictlybetter itself (not the target project).---# /strictlybetter:bench — the meta-benchmarkThe loop's own claim is testable (docs/10 §10.7). The runner is a separate program; this skill only drives it.```bash# Resolve the engine. RUN THIS BLOCK VERBATIM — do not substitute a path you guessed.# Order: ZCode's plugin root (ZCode also exports the legacy CLAUDE_PLUGIN_ROOT, so its own# var is checked first), Claude Code, Codex, an explicit checkout (SB_ROOT), the Claude Code# plugin cache (newest version), and LAST the working tree's git toplevel (a contributor# running inside the strictlybetter checkout). First one that holds scripts/sb.py wins.SB_PY=""SB_CACHE="$(find "$HOME/.claude/plugins/cache" -maxdepth 5 -path '*/strictlybetter/*/scripts/sb.py' 2>/dev/null | sort | tail -n 1)"for d in "${ZCODE_PLUGIN_ROOT:-}" "${CLAUDE_PLUGIN_ROOT:-}" "${CODEX_PLUGIN_ROOT:-}" "${SB_ROOT:-}" \ "${SB_CACHE%/scripts/sb.py}" "$(git rev-parse --show-toplevel 2>/dev/null)"; do [ -n "$d" ] && [ -f "$d/scripts/sb.py" ] && SB_PY="$d/scripts/sb.py" && breakdoneif [ -z "$SB_PY" ]; then echo "strictlybetter: engine not found — set SB_ROOT to your strictlybetter checkout" >&2 return 2 2>/dev/null || exit 2 # FAIL CLOSED: proceeding would run `python3 ""`fiSB_ROOT="${SB_PY%/scripts/sb.py}"SB_REPO="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"sb() { python3 "$SB_PY" --repo "$SB_REPO" "$@"; } # a FUNCTION, not a string: zsh does notSB=sb # word-split "$SB"; both shells run `sb`$SB --version | grep -q '^sb 1\.' || { echo "strictlybetter: engine did not answer (expected sb 1.x)" >&2; return 2 2>/dev/null || exit 2; } # major version only; exact patch level is not asserted```## 0 · Re-anchor (never trust conversational memory)```bash$SB status --json # the truth on disk$SB status --json | grep -q '"campaign": null' || $SB next # the cold-start brief, when a campaign exists```## 1 · Entry point```bashls "$SB_ROOT/bench/" 2>/dev/null || echo "bench runner not installed in this checkout"python3 "$SB_ROOT/bench/run_bench.py" --help```Follow the usage the runner prints; pass the user's arguments through unchanged. Do notinvent flags. Runs are long (each condition drives full campaigns on the fixture repos under`tests/fixtures/`); say so before starting and prefer running it in the background.## 2 · ResultsResults land in `$SB_ROOT/bench/results/` (one directory or file per run, written by therunner, never hand-edited). Print the path of the newest result and quote its summary tableverbatim. The two numbers that matter are the **false-accept rate** on a fresh holdout foreach condition (all walls vs naive) and **cost per accepted improvement**. If the walls donot reduce the false-accept rate at acceptable cost, say so plainly; the thesis isfalsifiable and the docs promise to report it either way.

Frequently asked questions