Benchmark the live setup
Run the repo's benchmark against this harness exactly as it is configured right now — extensions, skills and MCP servers included, nothing isolated.
From the repository root (/home/montimage/buildspace/m-bench), run:
./bench setup --harness pi --suite agentic-hard
- Add
-m <provider/model>only if you want a specific model; otherwise the run asks (or use BENCH_HARNESS_MODEL). - Add
--samples 2for a less noisy score; add--thinkingto test the reasoning mode. - The run takes several minutes per sample and executes model-generated code on this host.
When it finishes:
- Read the report path it prints (
results/<date>/REPORT-live*.md) and open it. - Relay the headline numbers (agent score, solve rate, mean calls vs par) and every bullet in the Suggestions section back to the user, verbatim where possible.
Note the caveat from the report: live mode keeps your real setup enabled, so if any extension or MCP server can call another model, those numbers are contaminated by that model.