# Running Test262 > Run the repo's canonical Test262 suite for parser/runtime conformance; never write ad hoc runners. - Skill: `socketdev/running-test262` (Agent Skill) - Install (CLI): `npx skillmds@latest add socketdev/running-test262` - Raw SKILL.md: https://api.skillmd.com/api/skills/socketdev/running-test262/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: socketdev (https://skillmd.com/u/socketdev) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/socketdev/running-test262 --- # running-test262 The fleet has multiple parsers + runtimes that conform to ECMA262 or to a TC39 proposal: - `ultrathink/packages/acorn/`: the JS parser, multiple lang ports (cpp/go/rust/typescript). - `ultrathink/packages/test262-parser-runner/`: the canonical shared runner package. - `socket-btm/packages/temporal-infra/`: Temporal-proposal C++ port. Every one of them ships its own `scripts/test262-*.mts` runner + an `unsupported-features` config. Running test262 by hand (downloading the suite, scanning the metadata blocks, running each test) is the wrong shape. The runners already encode the suite-traversal, the per-feature skip logic, the harness setup, and the result-aggregation. Always reach for the existing runner. ## Test262 submodule pin The fleet pins to a shared `tc39/test262` SHA. As of 2026-05-21 both ultrathink + socket-btm pin `7e115f46a`. When bumping in one repo, bump in the other so cross-fleet comparison stays apples-to-apples. Annotation lives in each repo's `.gitmodules` with the pattern `# test262-YYYY.MM.DD` (commit-date of the pinned SHA, enforced by the `gitmodules-comment-guard` hook). ## 🚨 Strict allowlist policy **An allowlist entry is ONLY for non-parser test fails.** Anything a parser should handle MUST NOT be allowlisted; it must be fixed in the parser. This is strict; the runners enforce it via design choices below. What counts as "non-parser": - **Unimplemented TC39 feature**: the proposal is at Stage 3+ but we haven't ported the grammar yet (decorators, source-phase imports). Goes in `test262-config/test262.unsupported-features` keyed on the TC39 feature name (NOT a test path). - **Runner / harness bug**: the test runner itself produces a false signal (e.g. async-throws semantics, error-name matching). Fix the runner, don't allow-list the symptom. - **Runtime-only test**: the test exercises a runtime API (`Reflect.*`, `Temporal.*`) that the parser-conformance run can't evaluate. The runners skip these by classification, not per-path allowlist. What does NOT count and must be fixed in the parser: - "Parser rejects valid input." Fix the parser. - "Parser accepts invalid input." Fix the parser. - "Parser produces wrong AST shape." Fix the parser. - "Cross-impl divergence: Rust + TS pass, Go fails." Fix Go. If you feel tempted to add a per-test-path allowlist entry, the answer is almost always "the parser needs fixing." The `unsupported-features` file is the only escape valve and it's feature-name-keyed by design. You can't sneak a parser bug past it. ## Canonical runners per repo | Repo | Runner | Skip config | | --------------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | | ultrathink/packages/acorn (multi-lane driver) | `test/test262-compare.mts` | per-lane runner config (inherits unsupported-features) | | ultrathink/packages/acorn (per-lane) | `lang//scripts/test262.mts` | `test262-config/test262.unsupported-features` (feature-name-keyed) | | ultrathink/packages/test262-parser-runner | `bin/test262-parser-runner.mts` | passed via flags | | socket-btm/packages/temporal-infra | `test/scripts/test262-temporal-runner.mts` | `test262-config/test262.allowlist` (Temporal-only path allowlist; reviewed manually for non-parser-fail justification) | ## Invocation patterns ### Multi-lane (recommended for cross-lane parity checks) ```bash cd packages/acorn # All 4 lanes, full suite node test/test262-compare.mts # Subset of lanes node test/test262-compare.mts --lane rust,go # All lanes, filtered to a single category node test/test262-compare.mts --include 'language/expressions/await' # Single test path, all lanes node test/test262-compare.mts test/language/statements/class/private-method.js ``` Lanes: `rust`, `go`, `cpp`, `typescript`. Flags forward to each per-lane runner. ### Single-lane ```bash # Per-lane direct invocation cd packages/acorn/lang/rust && node scripts/test262.mts cd packages/acorn/lang/go && node scripts/test262.mts cd packages/acorn/lang/cpp && node scripts/test262.mts cd packages/acorn/lang/typescript && node scripts/test262.mts # socket-btm temporal-infra cd socket-btm/packages/temporal-infra && node test/scripts/test262-temporal-runner.mts ``` ### Single-case debug Pass the test path positionally: ```bash # Single lane node scripts/test262.mts test/language/expressions/await/await-in-nested-function.js # All lanes node test/test262-compare.mts test/language/expressions/await/await-in-nested-function.js ``` ### Targeted filtering ```bash node scripts/test262.mts --include 'export' # regex on path node scripts/test262.mts --exclude 'surrogate' # regex on path node scripts/test262.mts --category module # named feature group node scripts/test262.mts --include 'class' --exclude 'async' ``` ### Vitest-integrated mode Each repo also wires a vitest test that wraps the runner. Useful for CI integration and selective re-runs: ```bash pnpm exec vitest run test/unit/test262.test.mts # ultrathink acorn pnpm exec vitest run test/unit/test262-temporal.test.mts # socket-btm temporal ``` ## Common failure modes - **Submodule missing.** The test262 suite is a git submodule. If the runner errors with "test262 suite not found", run `git submodule update --init --recursive`. - **Feature classification drift.** The runner uses each test's metadata block (`/*--- features: [...] ---*/`) to decide whether to run or skip. If a new TC39 feature is added upstream, classify it in the `unsupported-features` config first; do not let the runner silently pass tests for features the parser doesn't implement. - **"Allowlist drift": does NOT apply here.** The acorn lanes don't carry a per-test-path allowlist. If a test starts passing or failing, that's the parser's behavior; either the parser is correct and the test is correct (good), or one of them is wrong and that's a bug. - **Cross-fleet drift.** ultrathink and socket-btm should pin the same `tc39/test262` SHA. If you're investigating a flaky test, double-check both `.gitmodules` files first. ## Never write a homebrew runner The existing runners encode dozens of edge cases (strict-mode harness wrapping, async-throws semantics, error-name matching, the `negative.phase` distinction between parse vs early errors). Recreating that surface from scratch reliably misses cases. If you find yourself wanting to "just run a few test262 files by hand," reach for the runner with a filter arg instead. ## Reference - TC39 test262 spec: https://github.com/tc39/test262 - Each runner's source is the source of truth for invocation flags and exit-code conventions; cat the runner first if the invocation is unclear. - Strict allowlist policy + multi-lane behavior + `tc39/test262` pin date all encoded in this skill. Read this skill before touching either system.