# Run Tests

> Run tests and generate reference tests. All testing goes through `make test`, never `pytest` directly. Always load this skill before running tests.

- Skill: `ethereum/run-tests` (Agent Skill)
- Install (CLI): `npx skillmds@latest add ethereum/run-tests`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ethereum/run-tests/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: ethereum (https://skillmd.com/u/ethereum)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ethereum/run-tests

---


# Running tests

All testing is done with `make test`. Do not use `pytest` directly. When
possible, testing should be targeted to areas affected by the change rather than
running the full suite. When fixing issues, the targeted set should be run once
to collect failures, then individual failing tests should be re-run to verify
fixes rather than re-running the whole filtered set after each edit.

## Filter by name

Tests can be filtered by name via the `k=<string>` flag. This value is passed
directly to pytest with `-k` and therefore supports its features, such as
boolean operators (`and`, `or`, `not`). For example,
`make test k="gossip and valid"` is valid.

## Debug mode

`print()` output can be made visible with the `debug=true` flag. These messages
are not shown when tests run in parallel.

## Filter by fork

Tests can be limited to a single fork via the `fork=<fork>` flag. It is not
currently possible to specify multiple forks with the flag. The available forks
are the names of all directories in `./specs/` (except for `_features`) and all
directories in `./specs/_features/`.

## Filter by preset

Tests can be run against a preset via the `preset=<preset>` flag. There are
three presets: general, minimal, and mainnet. By default, `make test` only runs
minimal tests. The general tests cover things like BLS, KZG, and SSZ tests. The
minimal tests are fast to execute but do not cover the full range of possible
situations. The mainnet tests are very slow and should only be run in targeted
commands. Note that there are some tests which are only executed under a single
preset.

## Generate reference tests

Reference tests can be generated for clients to ensure compliance with the
specifications. Enable reference tests outputs with the `reftests=true` flag.
These are written to `./reftests` at the project root; this directory is
automatically created. Unlike `make test`, `reftests=true` runs all presets
(general, minimal, mainnet) by default; this can be overridden by using the
`preset=<preset>` flag. The framework deletes individual test case directories
before regenerating them, so targeted reruns will update existing reference
tests. If a test case function was deleted, `make test reftests=true` will not
delete its previously generated reference test.

## Track code coverage

Code coverage tracking can be enabled with the `coverage=true` flag. The results
are available as HTML for humans at `./tests/core/pyspec/.htmlcov/index.html`
and as JSON for robots at `./tests/core/pyspec/.htmlcov/coverage.json`.