Gating tests with @gate / @force-gate
Full reference: test/lib/gate/README.md.
This skill is the decision guide: which directive to reach for, the standard
conversion patterns, and how to verify.
Never write these — gate instead
| Anti-pattern |
Replacement |
it.skip('...') for a known failure |
// @gate <cond> (or @gate FIXME if no condition explains it) |
if (isNextDev) { test('skipped in dev mode', () => {}); return } |
// @force-gate prefetching (or !dev) on the describe |
(flagEnabled ? describe.skip : describe)(...) keyed on process.env |
// @force-gate <cond> (lazy) on the describe |
| Duplicating a fixture directory per flag state |
one fixture keyed on __NEXT_TEST_AXIS + a @gate/@force-gate |
Branching expectations on process.env.__NEXT_CACHE_COMPONENTS |
if (await gate((c) => c.cacheComponents)) (gate from next-test-utils) |
The skip patterns are fake-greens: nothing tells you when the bug they hide is
fixed. @gate still runs the body and fails the suite the day the "known
failure" starts passing, so stale workarounds get deleted instead of rotting.
Choosing the directive
Ask what kind of difference you're encoding:
- A behavior change — both states assert something meaningful. Don't
gate the test at all: fork inside the body with the runtime
gate() —
same condition registry, no inversion — which pinpoints exactly what
differs, and also covers it.each, where a pragma cannot attach:
if (await gate((c) => c.cacheComponents)) { ... } else { ... }. It
mirrors React's gate(flags => ...); a pragma expression string works
too (await gate('cacheComponents && !dev')). A suite-level pragma is
too coarse here — it hides what is different between the states.
- A flag that changes the behavior of existing surface
(
cacheComponents, optimisticRouting) and the suite is written for one
state. // @gate <cond> on the test or describe. The body runs; a
false condition inverts the expectation (failure absorbed, a pass fails as
"stale gate"). The off state fails for a meaningful reason — the behavior
differs — so a pass is real information: the gate is stale, delete it.
- A new API — the off state proves nothing. Typically
// @force-gate <cond> (lazy) on the describe. An API that throws when
its flag is off — or is inert, like useOffline(), which compiles to a
hook that always reports online — can only fail vacuously (often slowly,
by timing out), and browser e2e time is considerable, so skip the run
(and the fixture build) instead of paying for it. Working example:
test/e2e/app-dir/use-offline/. This is discretion, not a rule: when the
flag changes behavior the suite can observe, the off state is meaningful
and @gate buys the staleness check.
@force-gate <cond> also when running the body is impossible, not
merely failing: prefetching is off in dev, deploy has no local build
output, the fixture cannot even build under the condition.
- Static condition (
!dev, bundler…) → real Jest ○ skipped at
collection.
- Lazy condition on a
describe → the fixture build is skipped when
false; tests report passed-with-⚠ skipped by @force-gate (Jest cannot
skip at runtime). Build-skipping covers start/dev suites where
nextTestSetup owns the build — not skipStart suites, not deploy.
- Pragmas stack: a common pair is a static
// @force-gate prefetching plus
a lazy // @gate <flag> on the same describe.
Is the off-state run worth its cost?
Browser e2e time is not free, so weigh what the gated-off run buys. For a
behavior flag it usually replaces a run that was already being paid for — a
fixture that pins its flags runs identically with and without the axis set,
so keying the flag on an axis converts a redundant duplicate into coverage —
and it is what proves a pass isn't vacuous: a test that passes with the
feature off wasn't testing the feature. Absorbed failures also fail fast, so
the off-state run is cheaper than it sounds. For a new API the calculus
flips: the off state can only throw, which proves nothing, so use a lazy
// @force-gate <flag> on the describe — the fixture build is skipped
too, so the off state costs almost nothing.
Conditions
Every name in a pragma must be declared in test/lib/gate/conditions.ts
(typos fail the suite at collection). Two tiers:
- static — the run's shape:
dev, start, deploy, mode, turbopack,
rspack, webpack, bundler, react18, wasm, ci; specialized CI
variants adapter, standaloneOutput, turbopackDev, and turbopackBuild;
plus the always-false FIXME/TODO.
prod and prefetching are semantic aliases for !dev — prefer the name
that states why the suite cannot run.
- lazy — a predicate over the fixture's resolved
next.config
(cacheComponents, ppr, useOffline, output, …).
Adding one is a two-line change; follow the guidance at the top of
conditions.ts. The rule that matters: lazy conditions read the resolved
config, never process.env — env vars don't survive config resolution
(__NEXT_CACHE_COMPONENTS only applies when the fixture doesn't set
cacheComponents itself, and resolution implies flags the fixture never
mentions).
Pattern: cover both states of an experimental flag
Instead of pinning a flag on (which makes the plain and axis runs identical),
key it on a test axis and gate the suite. Axes are lettered (A, B, …) —
a fixed enumeration, not a boolean and not a sharding bucket. Key the flag so
it is enabled by default — then the suite exercises the feature in plain
local runs with no special env, and the axis run covers the off state:
// next.config.js — pin every dimension except the one under test
const nextConfig = {
cacheComponents: true,
experimental: {
concurrentRouterQueue: process.env.__NEXT_TEST_AXIS !== 'A',
},
}
// @gate concurrentRouterQueue
it('fails loudly on link navigation', async () => { ... })
The plain run exercises the feature; the axis-A run covers the off state —
the gated tests are expected to fail there, and the suite fails the day they
start passing. Working example: test/e2e/app-dir/concurrent-router-queue/
(tests whose expectations hold in both states stay ungated). The same keying
pairs with a lazy @force-gate when the off state proves nothing —
test/e2e/app-dir/use-offline/ — which skips the redundant axis run (build
included) instead of covering it. Axis A aliases __NEXT_CACHE_COMPONENTS
for now (see scripts/run-jest.sh) — fine, because these fixtures pin
cacheComponents explicitly, so that run's env default is a no-op for them.
Keep exactly one flag varying per fixture. A red shard must attribute to a
single dimension.
Pitfalls
- A pragma the transform can't attach is a hard error: a blank line
between pragma and
it(, it.each/it.failing, or a pragma inside a
JSDoc block. Prose comments must not begin with @gate. A pragma on a
skipped test (it.skip, xit, …) errors as ambiguous — remove the skip or
the pragma. A skip without a pragma is respected.
- A
describe-level gate does not reach it.each tests.
- Gated-false bodies that fail by stalling waste the full Jest timeout —
and under a lazy gate they fail the suite anyway (the runtime inversion
only absorbs thrown errors; a static gate rides Jest's native
test.failing, which does absorb timeouts). Bodies that fail via retry()
timeouts also make the off-state run slow; a fast first assertion is worth
having.
- Failures cascade in the off state: an absorbed failure mid-body skips the
body's cleanup (e.g. a browser context left offline), so later tests may
fail for cascade reasons. Acceptable for a tripwire, but don't puzzle over
the individual failure messages in a gated-off run.
afterEach failures (e.g. redbox matchers) are not gated, only the body is.
Hooks under a false lazy @force-gate are the exception: they are skipped
with the suite instead of running against a fixture that was never booted.
jest.retryTimes(1) on non-dev CI means a flaky gated-false test passes
whenever it happens to fail; the tripwire is only deterministic for
deterministic tests.
- Gated titles are unchanged in the Jest output; the
⚠ gated test failed as expected log line is the only signal.
pragma-transform.js bails out early on files containing neither @gate
nor @force-gate as substrings — keep both checks if you touch it.
Verify a gated suite in every state it can run in
# plain run (flag on): expect normal passes, no warnings
NEXT_SKIP_ISOLATE=1 pnpm test-start-webpack test/e2e/app-dir/<suite>/<suite>.test.ts
# axis run (flag off): expect `⚠ gated test failed as expected (@gate …)`
__NEXT_TEST_AXIS=A NEXT_SKIP_ISOLATE=1 pnpm test-start-webpack test/e2e/app-dir/<suite>/<suite>.test.ts
# dev (static @force-gate !dev): expect `○ skipped` at collection, no fixture boot
NEXT_SKIP_ISOLATE=1 pnpm test-dev-webpack test/e2e/app-dir/<suite>/<suite>.test.ts
A suite with a lazy @force-gate on the describe should additionally show
skipping build behavior (no next build) in the state where the condition
is false.
Unit tests for the infrastructure itself: pnpm test-unit test/unit/gate/.
Related skills
$flags — adding the experimental flag itself (config-shared, schema,
define-env)
$router-act — the prefetch-timing patterns most gated suites also use
1---
2name: gate-tests
3description: >
4 How to use the `@gate` / `@force-gate` test directives instead of `it.skip`
5 or fake-green skip patterns. Use when a test is known-failing under some
6 test-matrix dimension (dev mode, a bundler, an experimental flag like
7 cacheComponents), when converting `if (isNextDev) return` guards or
8 env-var `describe.skip` branches, when adding a condition to
9 test/lib/gate/conditions.ts, or when keying a fixture's experimental flag
10 on a __NEXT_TEST_AXIS letter. Covers directive choice, condition tiers,
11 the test-axis fixture pattern, pitfalls, and verification commands.
12user-invocable: false
13metadata:
14 source: https://skillmd.com/skills/vercel/gate-tests
15 internal: true
16---
17
18# Gating tests with `@gate` / `@force-gate`
19
20Full reference: [`test/lib/gate/README.md`](../../../test/lib/gate/README.md).
21This skill is the decision guide: which directive to reach for, the standard
22conversion patterns, and how to verify.
23
24## Never write these — gate instead
25
26| Anti-pattern | Replacement |
27| ---------------------------------------------------------------------- | --------------------------------------------------------------------------- |
28| `it.skip('...')` for a known failure | `// @gate <cond>` (or `@gate FIXME` if no condition explains it) |
29| `if (isNextDev) { test('skipped in dev mode', () => {}); return }` | `// @force-gate prefetching` (or `!dev`) on the `describe` |
30| `(flagEnabled ? describe.skip : describe)(...)` keyed on `process.env` | `// @force-gate <cond>` (lazy) on the `describe` |
31| Duplicating a fixture directory per flag state | one fixture keyed on `__NEXT_TEST_AXIS` + a `@gate`/`@force-gate` |
32| Branching expectations on `process.env.__NEXT_CACHE_COMPONENTS` | `if (await gate((c) => c.cacheComponents))` (`gate` from `next-test-utils`) |
33
34The skip patterns are fake-greens: nothing tells you when the bug they hide is
35fixed. `@gate` still runs the body and fails the suite the day the "known
36failure" starts passing, so stale workarounds get deleted instead of rotting.
37
38## Choosing the directive
39
40Ask what kind of difference you're encoding:
41
421. **A behavior change — both states assert something meaningful.** Don't
43 gate the test at all: fork inside the body with the runtime `gate()` —
44 same condition registry, no inversion — which pinpoints exactly what
45 differs, and also covers `it.each`, where a pragma cannot attach:
46 `if (await gate((c) => c.cacheComponents)) { ... } else { ... }`. It
47 mirrors React's `gate(flags => ...)`; a pragma expression string works
48 too (`await gate('cacheComponents && !dev')`). A suite-level pragma is
49 too coarse here — it hides _what_ is different between the states.
502. **A flag that changes the behavior of existing surface**
51 (`cacheComponents`, `optimisticRouting`) **and the suite is written for one
52 state.** `// @gate <cond>` on the test or `describe`. The body runs; a
53 false condition inverts the expectation (failure absorbed, a pass fails as
54 "stale gate"). The off state fails for a meaningful reason — the behavior
55 differs — so a pass is real information: the gate is stale, delete it.
563. **A new API — the off state proves nothing.** Typically
57 `// @force-gate <cond>` (lazy) on the `describe`. An API that throws when
58 its flag is off — or is inert, like `useOffline()`, which compiles to a
59 hook that always reports online — can only fail vacuously (often slowly,
60 by timing out), and browser e2e time is considerable, so skip the run
61 (and the fixture build) instead of paying for it. Working example:
62 `test/e2e/app-dir/use-offline/`. This is discretion, not a rule: when the
63 flag changes behavior the suite can observe, the off state is meaningful
64 and `@gate` buys the staleness check.
654. **`@force-gate <cond>` also when running the body is impossible**, not
66 merely failing: prefetching is off in dev, deploy has no local build
67 output, the fixture cannot even build under the condition.
68 - Static condition (`!dev`, `bundler`…) → real Jest `○ skipped` at
69 collection.
70 - Lazy condition on a `describe` → the fixture **build is skipped** when
71 false; tests report passed-with-`⚠ skipped by @force-gate` (Jest cannot
72 skip at runtime). Build-skipping covers `start`/`dev` suites where
73 `nextTestSetup` owns the build — not `skipStart` suites, not deploy.
745. Pragmas stack: a common pair is a static `// @force-gate prefetching` plus
75 a lazy `// @gate <flag>` on the same `describe`.
76
77### Is the off-state run worth its cost?
78
79Browser e2e time is not free, so weigh what the gated-off run buys. For a
80behavior flag it usually replaces a run that was already being paid for — a
81fixture that pins its flags runs identically with and without the axis set,
82so keying the flag on an axis converts a redundant duplicate into coverage —
83and it is what proves a pass isn't vacuous: a test that passes with the
84feature off wasn't testing the feature. Absorbed failures also fail fast, so
85the off-state run is cheaper than it sounds. For a new API the calculus
86flips: the off state can only throw, which proves nothing, so use a lazy
87`// @force-gate <flag>` on the `describe` — the fixture build is skipped
88too, so the off state costs almost nothing.
89
90## Conditions
91
92Every name in a pragma must be declared in `test/lib/gate/conditions.ts`
93(typos fail the suite at collection). Two tiers:
94
95- **static** — the run's shape: `dev`, `start`, `deploy`, `mode`, `turbopack`,
96 `rspack`, `webpack`, `bundler`, `react18`, `wasm`, `ci`; specialized CI
97 variants `adapter`, `standaloneOutput`, `turbopackDev`, and `turbopackBuild`;
98 plus the always-false `FIXME`/`TODO`.
99 `prod` and `prefetching` are semantic aliases for `!dev` — prefer the name
100 that states _why_ the suite cannot run.
101- **lazy** — a predicate over the fixture's _resolved_ `next.config`
102 (`cacheComponents`, `ppr`, `useOffline`, `output`, …).
103
104Adding one is a two-line change; follow the guidance at the top of
105`conditions.ts`. The rule that matters: **lazy conditions read the resolved
106config, never `process.env`** — env vars don't survive config resolution
107(`__NEXT_CACHE_COMPONENTS` only applies when the fixture doesn't set
108`cacheComponents` itself, and resolution implies flags the fixture never
109mentions).
110
111## Pattern: cover both states of an experimental flag
112
113Instead of pinning a flag on (which makes the plain and axis runs identical),
114key it on a test axis and gate the suite. Axes are lettered (`A`, `B`, …) —
115a fixed enumeration, not a boolean and not a sharding bucket. Key the flag so
116it is **enabled by default** — then the suite exercises the feature in plain
117local runs with no special env, and the axis run covers the off state:
118
119```js
120// next.config.js — pin every dimension except the one under test
121const nextConfig = {
122 cacheComponents: true,
123 experimental: {
124 concurrentRouterQueue: process.env.__NEXT_TEST_AXIS !== 'A',
125 },
126}
127```
128
129```ts
130// @gate concurrentRouterQueue
131it('fails loudly on link navigation', async () => { ... })
132```
133
134The plain run exercises the feature; the axis-A run covers the off state —
135the gated tests are expected to fail there, and the suite fails the day they
136start passing. Working example: `test/e2e/app-dir/concurrent-router-queue/`
137(tests whose expectations hold in both states stay ungated). The same keying
138pairs with a lazy `@force-gate` when the off state proves nothing —
139`test/e2e/app-dir/use-offline/` — which skips the redundant axis run (build
140included) instead of covering it. Axis `A` aliases `__NEXT_CACHE_COMPONENTS`
141for now (see `scripts/run-jest.sh`) — fine, because these fixtures pin
142`cacheComponents` explicitly, so that run's env default is a no-op for them.
143
144**Keep exactly one flag varying per fixture.** A red shard must attribute to a
145single dimension.
146
147## Pitfalls
148
149- A pragma the transform can't attach is a **hard error**: a blank line
150 between pragma and `it(`, `it.each`/`it.failing`, or a pragma inside a
151 JSDoc block. Prose comments must not begin with `@gate`. A pragma on a
152 skipped test (`it.skip`, `xit`, …) errors as ambiguous — remove the skip or
153 the pragma. A skip without a pragma is respected.
154- A `describe`-level gate does not reach `it.each` tests.
155- Gated-false bodies that fail by _stalling_ waste the full Jest timeout —
156 and under a lazy gate they fail the suite anyway (the runtime inversion
157 only absorbs thrown errors; a static gate rides Jest's native
158 `test.failing`, which does absorb timeouts). Bodies that fail via `retry()`
159 timeouts also make the off-state run slow; a fast first assertion is worth
160 having.
161- Failures cascade in the off state: an absorbed failure mid-body skips the
162 body's cleanup (e.g. a browser context left offline), so later tests may
163 fail for cascade reasons. Acceptable for a tripwire, but don't puzzle over
164 the individual failure messages in a gated-off run.
165- `afterEach` failures (e.g. redbox matchers) are not gated, only the body is.
166 Hooks under a false lazy `@force-gate` are the exception: they are skipped
167 with the suite instead of running against a fixture that was never booted.
168- `jest.retryTimes(1)` on non-dev CI means a _flaky_ gated-false test passes
169 whenever it happens to fail; the tripwire is only deterministic for
170 deterministic tests.
171- Gated titles are unchanged in the Jest output; the
172 `⚠ gated test failed as expected` log line is the only signal.
173- `pragma-transform.js` bails out early on files containing neither `@gate`
174 nor `@force-gate` as substrings — keep both checks if you touch it.
175
176## Verify a gated suite in every state it can run in
177
178```sh
179# plain run (flag on): expect normal passes, no warnings
180NEXT_SKIP_ISOLATE=1 pnpm test-start-webpack test/e2e/app-dir/<suite>/<suite>.test.ts
181
182# axis run (flag off): expect `⚠ gated test failed as expected (@gate …)`
183__NEXT_TEST_AXIS=A NEXT_SKIP_ISOLATE=1 pnpm test-start-webpack test/e2e/app-dir/<suite>/<suite>.test.ts
184
185# dev (static @force-gate !dev): expect `○ skipped` at collection, no fixture boot
186NEXT_SKIP_ISOLATE=1 pnpm test-dev-webpack test/e2e/app-dir/<suite>/<suite>.test.ts
187```
188
189A suite with a lazy `@force-gate` on the `describe` should additionally show
190`skipping build` behavior (no `next build`) in the state where the condition
191is false.
192
193Unit tests for the infrastructure itself: `pnpm test-unit test/unit/gate/`.
194
195## Related skills
196
197- `$flags` — adding the experimental flag itself (config-shared, schema,
198 define-env)
199- `$router-act` — the prefetch-timing patterns most gated suites also use