# Motoko Benchmarks Generation

> How to write benchmarks in Motoko using bench‑helper. Covers project setup (mops.toml), bench file layout in bench/*.bench.mo, the Bench.Schema rows/cols model, and safe patterns for encode/decode, hashing, crypto, and allocation benches.

- Skill: `research-ag/motoko-benchmarks-generation` (Agent Skill)
- Install (CLI): `npx skillmds@latest add research-ag/motoko-benchmarks-generation`
- Raw SKILL.md: https://api.skillmd.com/api/skills/research-ag/motoko-benchmarks-generation/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: research-ag (https://skillmd.com/u/research-ag)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/research-ag/motoko-benchmarks-generation

---


# Motoko Benchmarks with bench‑helper

## What This Is

`bench-helper` is a tiny Motoko library that standardizes how to write benchmarks.
You describe a benchmark using a small schema (name, rows, cols), provide a `run(row, col)`
function, and return a versioned bench record. Each file under `bench/*.bench.mo` defines one
benchmark module. A runner can then discover and execute all benches consistently.

## Prerequisites

mops.toml (add dependencies and toolchain)

If you already have a `mops.toml`, just add `bench-helper` under `[dev-dependencies]`.
If your project still uses `mo:base` instead of `mo:core`, you can keep it — benches themselves can be written with
`mo:base` without affecting your runtime canisters.

**Make sure `[optimize]` is configured.** `mops bench` only runs Binaryen's `wasm-opt` if `mops.toml` has an
`[optimize]` section — even an empty one activates it (default `level = "O3"`). Without it, `mops bench` measures
unoptimized Wasm, which doesn't reflect what a production canister actually runs. `[optimize]` requires a pinned
`[toolchain] wasm-opt` version, e.g.:

```toml
[optimize]

[toolchain]
wasm-opt = "132"   # mops toolchain use wasm-opt <version>
```

The `motoko-mops-package-maintenance` skill adds this automatically for packages with benchmarks during a
maintenance pass — if it hasn't run yet, add it yourself before trusting bench numbers.

## Directory & File Conventions

- Put benches under `bench/` at the repo root.
- Name files with the suffix `.bench.mo`, one benchmark per file; for example: `bench/base64.bench.mo`.
- Each bench file is a Motoko `module { ... }` that exposes a single `public func init() : Bench.V1` function.
- Inside `init`, construct a `Bench.Schema` and return `Bench.V1(schema, run)` where
  `run : (rowIndex : Nat, colIndex : Nat) -> ()` performs the measured operation.

Minimal skeleton

```motoko
import Array "mo:core/Array";
import Bench "mo:bench-helper";

module {
  /// One benchmark per file; `mops bench` calls `init()` to obtain the
  /// schema and the run function.
  public func init() : Bench.V1 {
    let schema : Bench.Schema = {
      name = "My bench";
      description = "What this bench measures";
      rows = ["size 16", "size 64", "size 256"]; // input categories
      cols = ["operation A", "operation B"];     // operations to measure
    };

    // Prepare inputs once in `init` so they are not re-created on every call.
    // `Array.tabulate` returns an immutable `[Nat8]`; use `VarArray.repeat`
    // (or `VarArray.tabulate`) if you need a `[var Nat8]` instead.
    let inputs : [[Nat8]] = [
      Array.tabulate<Nat8>(16, func _ = 0),
      Array.tabulate<Nat8>(64, func _ = 0),
      Array.tabulate<Nat8>(256, func _ = 0),
    ];

    // Build a table of routines: routines[ri][ci] : () -> ()
    let routines : [[() -> ()]] = Array.tabulate<[() -> ()]>(
      schema.rows.size(),
      func(ri) {
        let input = inputs[ri]; // capture precomputed input
        [
          func() { ignore input.size() }, // operation A @ inputs[ri]
          func() { ignore input[0] },     // operation B @ inputs[ri]
        ];
      },
    );

    // `Bench.V1` is a *class* (instance type), not a record alias.
    // The runner calls the second argument many times; keep it tiny
    // and branch-free.
    Bench.V1(schema, func(ri : Nat, ci : Nat) = routines[ri][ci]());
  };
};
```

Notes:
- The `Schema` field is `cols`, not `columns`. Same for `rows.size()` — you
  must say `schema.rows.size()` (the bare `rows` identifier is not in scope).
- `mo:core/Array` has no `init` function. To build an immutable `[T]`, use
  `Array.tabulate<T>(size, gen)` or `Array.repeat<T>(val, size)`. For a
  mutable `[var T]`, import `VarArray "mo:core/VarArray"` and use
  `VarArray.tabulate<T>(size, gen)` or `VarArray.repeat<T>(val, size)`;
  annotate such inputs as `[var T]` (e.g. `[[var Nat8]]` for a table of them).
- If you're not using `mo:core`, replace `mo:core` imports with `mo:base`.
  Note that `mo:base/Array` covers both immutable and mutable arrays — there
  is no separate `VarArray` module. The interface is not identical to
  `mo:core`; for the Array API the relevant equivalences are:
    - `mo:core` `Array.tabulate(size, gen)` → `mo:base` `Array.tabulate(size, gen)`
    - `mo:core` `Array.repeat(val, size)` → `mo:base` `Array.freeze(Array.init(size, val))` (no direct immutable `repeat`)
    - `mo:core` `VarArray.tabulate(size, gen)` → `mo:base` `Array.tabulateVar(size, gen)`
    - `mo:core` `VarArray.repeat(val, size)` → `mo:base` `Array.init(size, val)` (args reversed)
  See the `base-core-migration` skill for the full mapping.

## How It Works

- Schema
    - `name` and `description` describe the bench.
    - `rows` enumerate different operations or variants you measure (e.g., "encode", "decode").
    - `cols` enumerate different input categories (e.g., message sizes).
- Runner contract
    - You return a versioned record `Bench.V1(schema, run)`; the runner calls `run(rowIndex, colIndex)` many times to
      record timings.
    - Side effects/results inside `run` should be consumed (e.g., `ignore ...`) to prevent dead‑code elimination.

## Common Pitfalls

1. Rows/cols mismatch
    - Ensure your `routines` table has dimensions `rows.size() x cols.size()`. If you add or remove a row/col label, update how you build `routines`; otherwise some cells will be no‑ops.
2. Doing expensive setup inside `run`
    - Generate inputs once in `init` and capture them in closures. Only do the core operation in `run`.
3. Forgetting to consume results
    - Use `ignore` to consume return values; otherwise the compiler might drop the call as dead code.
4. Non‑determinism and timing noise
    - Keep `run` free of logging/printing and random allocation; keep GC pressure comparable across rows.
5. Repeated runs and mutating operations
    - `bench-helper` calls `run(ri, ci)` many times per cell to gather
      timing samples. If the measured operation mutates the captured input
      (e.g. an in-place sort, an in-place hash update), the second call
      onwards sees the *post-operation* state — for an in-place sort, the
      already-sorted array from the previous call; for an in-place hash
      update, the accumulator left behind by the previous call. Either
      rebuild the input inside `run` (and accept the allocation cost as
      part of the measurement) or document that the bench measures
      repeated runs on a mutated/post-operation state rather than on a
      freshly prepared input.

## More examples

bench-helper reference benches: https://github.com/research-ag/bench-helper/tree/main/bench

## Verify It Works

The exact runner/command may vary depending on your environment. After adding `bench-helper` to `mops.toml` and writing
`.bench.mo` files:

- Ensure your project resolves dependencies:
  ```bash
  mops install
  ```
- Run benchmarks:
  ```bash
  mops bench
  ```
- Consult the `bench-helper` package README for the latest recommended runner command for your toolchain/version.