Benchmark Run Sizing

Analyzes the Hardhat regression benchmark history (the hardhat-benchmark-results repo's data.js) and recommends how many times each benchmark should run. It reads every commit's individual run times, computes each benchmark's per-commit coefficient of variation, removes outliers, takes the 95th-percentile CV, and sizes run counts for target noise levels (sigma = 3%, 1.5%, 1%, i.e. regression alert limits of 10% / 5% / 3%). Use when asked to right-size benchmark run counts, justify or revisit run counts, assess benchmark variance/noise, or decide whether to add or remove runs after changing the regression alert threshold.

nomicfoundation Updated
Picture from the README

File contents

nomicfoundation/hardhat commit c7ff40024c

Frequently asked questions

npx get-skill add nomicfoundation/benchmark-run-sizing