Wpt Update

WPT Update

Context

WPT (Web Platform Tests) is an open-source test suite for assessing runtime conformance with Web standards. workerd keeps configuration under src/wpt because some tests are not applicable or cover behavior workerd does not yet implement.

A request to bump or update WPT includes both the dependency update and triage unless the user explicitly requests a bump-only change.

Scope

Allowed edits are dependency metadata produced by the updater and WPT config files under src/wpt. Do not fix runtime conformance bugs during a WPT update. If correct triage requires runtime, harness, build-rule, patch, or upstream changes, report the required follow-up but don't make changes unless the user expands the scope.

Preflight

1. Determine whether workerd is standalone or a submodule and read all applicable AGENTS.md files.
2. Inspect the worktree and preserve unrelated changes.
3. Load ts-style before editing TypeScript config files.

Bumping WPT

From the workerd root, run:

./build/deps/update_wpt.py

The updater selects the newest published wpt-* release from cloudflare/workerd-tools. A normal update changes only:

• build/deps/shared_deps.jsonc
• build/deps/gen/shared_deps.MODULE.bazel

Verify that the new tag appears consistently in freeze_version, strip_prefix, and the archive URL, and that a new SHA-256 was generated. Do not hand-edit generated dependency output.

The updater writes the manifest before downloading and regenerating metadata. If it fails, do not continue with inconsistent files. Resolve the failure, rerun the updater, and verify both files.

The generic dependency updater may mention an unrelated available release because workerd-tools contains several tools. Determine success from the selected WPT tag and resulting diff.

If the updater reports that WPT is current, stop for a bump-only request. Keep the dependency changes separate from triage changes. If commits were explicitly requested, commit the dependency update before moving on. Otherwise, continue without committing.

Triaging a WPT update

For a triage-only request, first verify that a pin change exists relative to the agreed base. Read the old and new wpt-<sha> values from the dependency diff. The suffix identifies the corresponding upstream WPT commit.

Use pinned SHAs for all upstream research. Do not inspect files from upstream main. GitHub's compare API truncates large WPT updates, so use the complete diff when necessary:

https://github.com/web-platform-tests/wpt/compare/<old-sha>...<new-sha>.diff

Filter upstream changes to the WPT directories mapped by src/wpt/BUILD.bazel. For an individual failure, inspect the changed file, its commits, the associated pull request, and any linked specification change.

Running WPT

The commands below show the correct repository labels. Add all flags required by the active test-hygiene instructions.

For standalone workerd, run from the workerd root:

bazel test //src/wpt/... --test_size_filters=

For a submodule checkout, run from the parent repository root:

bazel test @workerd//src/wpt/... --test_size_filters=

Clearing test_size_filters ensures that enormous WPT targets are included. The complete pattern runs default and all-autogates variants plus ESLint and TypeScript checks.

Use an affected suite target while iterating. Include the required @ suffix. For standalone workerd:

bazel test //src/wpt:<suite>@ \
  --test_size_filters= \
  --test_output=errors

For a submodule checkout:

bazel test @workerd//src/wpt:<suite>@ \
  --test_size_filters= \
  --test_output=errors

Do not use test-case filter arguments. Resolve the log root with bazel info bazel-testlogs or use the path printed by Bazel. Parent-repository logs are nested beneath the external workerd repository.

Search logs first for these messages instead of reading them in full:

Missing test configuration
unexpectedly failed
unexpectedly succeeded
Please update the test config
Test file ... not found

Handling test suite-level changes

A suite is a discovered WPT JavaScript file. Its config key is the exact, case-sensitive path relative to the wpt_directory declared in src/wpt/BUILD.bazel.

Discovery excludes non-JavaScript files and JavaScript files whose paths contain .tentative. or /tentative/.

Config keys MUST remain in ascending order as enforced by ESLint's sort-keys rule. Compare the complete raw key: punctuation affects ordering, so - sorts before .. Do not place a new key based only on visual grouping.

1. Check for renames or moves before treating files as additions and deletions. Transfer existing configuration across genuine renames.
2. Remove config keys for deleted suites.
3. Add each new suite as a sorted blank entry so its behavior can be observed:

'path/to/example.any.js': {},

Discovery includes JavaScript helper files unless their immediate parent is resources. Helpers outside those directories may need explicit omitted configuration even when they register no tests.

Re-run the complete WPT set after reconciling suite keys. Repeat until there are no missing or stale configuration errors.

After every config edit, run the aggregate //src/wpt:wpt-all@tsproject@eslint target with the repository prefix required by the checkout. Do not rely only on suite-specific ESLint targets.

Handling test changes

A subtest is an individual named test() or promise_test() within a suite.

1. Check whether removed and added names represent a renamed subtest.
2. When an expected failure now passes, determine whether workerd improved or upstream renamed, removed, or weakened the test before removing the expectation.
3. Investigate every new or newly failing subtest before classifying it.

Re-run affected suites while iterating, then rerun the complete WPT set until there are no unexpected failures or successes.

Analyzing failing tests

For each unexpected result:

1. Read the exact subtest failure and stack.
2. Inspect the upstream file at the new pinned SHA.
3. Identify the commit and specification change that introduced it.
4. Inspect the relevant workerd implementation or dependency.
5. Decide whether the test is relevant, unsafe to run, or inapplicable.
6. Record the current cause and any out-of-scope fix direction.
7. Compare default and all-autogates behavior before using one shared entry.

Do not classify an uncertain result merely to make the target pass. Runtime fixes are out of scope; explain the likely fix in the report instead.

META and load failures

META scripts and top-level code execute before subtests are registered. An expectedFailures entry cannot classify a missing resource or evaluation error at this stage. Partial disabledTests or omittedTests arrays cannot avoid it either.

Whole-file disabledTests: true and omittedTests: true skip evaluation before META imports. Use either only when its classification is independently correct. Do not omit a relevant suite merely because harness support is missing.

Classifications

• expectedFailures: relevant subtests execute but currently fail assertions because workerd is non-conformant.
• disabledTests: relevant tests cannot run reliably or safely, such as hangs, crashes, state corruption, or harness limitations.
• omittedTests: tests are not applicable to workerd and are excluded from coverage.

Each property accepts true for a whole suite or an array of exact strings and regular expressions. Prefer exact subtest names. Use a regex only when it is narrow, anchored where practical, and checked against every intended name. Do not use broad regexes or whole-suite true merely to make a target pass.

Every classification requires a comment describing the current technical reason. Source comments must describe current behavior, not the investigation or history that produced the entry. Commit messages and reports may include the additional evidence and history.

The harness reports unmatched expectedFailures, but a regex can still hide partially stale cases. It does not report stale disabled or omitted array entries. Audit those manually against renamed and removed upstream subtests.

Verification

Before completion:

1. Remove temporary only settings.
2. Confirm there are no missing or stale suite keys.
3. Confirm there are no unexpected failures or successes.
4. Manually audit changed disabled and omitted selectors.
5. Run the complete WPT target set using the applicable checkout command.
6. Require Bazel to report Build completed successfully.
7. Confirm the final Executed N out of N tests: N tests pass summary covers every selected target.
8. Do not treat an interrupted or partially analyzed invocation as success.
9. Run formatting and whitespace checks from the workerd root:

python3 tools/cross/format.py --check git
git diff --check

Continuous improvement

Re-read the SKILL.md file. If there are any details that weren't clear or that you had to look up separately, edit the file to include them. Information that concerns how the WPT test framework works within workerd is valuable, but refrain from adding info about specific situations you encountered during a WPT update, as this is transient and not likely to be relevant to future WPT updates.

Final report

Report:

• The old and new WPT tags and pinned upstream comparison URL.
• Added, removed, and renamed suites.
• Expectations added or removed.
• Each classification and its durable technical reason.
• Relevant upstream and workerd source locations.
• Proposed out-of-scope fixes and unresolved items.
• Exact validation commands and final executed counts.
• Final Git state.

Git handoff

Keep the generated dependency bump and WPT triage in separate commits.