# Benchmark

> Run benchmark suites and manage policy evolution — create challengers, compare against champions, promote or rollback policies.

- Skill: `hmbown/benchmark` (Agent Skill)
- Install (CLI): `npx skillmds@latest add hmbown/benchmark`
- Raw SKILL.md: https://api.skillmd.com/api/skills/hmbown/benchmark/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: Hmbown (https://skillmd.com/u/hmbown)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/hmbown/benchmark

---


# Autodialectics Benchmark & Evolve

Use this skill to benchmark policies and drive champion/challenger evolution.

## Prerequisites

`autodialectics-mcp` must be on PATH (`pip install autodialectics`).

## MCP Workflow

### Benchmarking

1. **`benchmark(suite_dir?, policy_id?)`** — run the benchmark suite against a policy. Returns case-by-case results with scores and decisions.

### Policy Evolution

2. **`evolve_policy(use_gepa?)`** — analyze recent benchmark reports and create a challenger policy. Set `use_gepa: false` to skip the GEPA optimizer (simpler heuristic fallback).
3. **`promote_policy(policy_id)`** — promote a challenger to champion if comparison rules allow.
4. **`rollback_policy()`** — revert to the previous champion if the current one regresses.

## CLI Fallback

```bash
autodialectics benchmark
autodialectics evolve
autodialectics promote <policy_id>
autodialectics rollback
```

## Typical Evolution Cycle

```
benchmark → evolve_policy → benchmark (with challenger) → compare → promote or rollback
```

1. Run benchmarks with the current champion to establish a baseline.
2. Evolve a challenger from the benchmark reports.
3. Run the same benchmarks with the challenger's policy ID.
4. Compare results. Promote if the challenger wins; rollback if it regresses.

## Guidance

- Never claim a policy is better without benchmark evidence from the same suite.
- When reporting benchmark results, include: total cases, pass/fail/revise counts, mean overall score, mean slop composite.
- If `evolve_policy` returns `no_reports`, run benchmarks first to generate data.
- Promotion can be denied by comparison rules — check the response status.

## Arguments

If the user passes a suite directory after `/autodialectics:benchmark`, use it as the benchmark suite path.