Review library-bulk-update PRs
These PRs are created by the scheduled compatibility workflow and are expected to update supported library versions by changing metadata/**/index.json. They may also refresh the generated root COVERAGE.md when tested-version changes affect the coverage table.
Historical Review Logic
Previous reviews follow a simple pattern:
- Approve silently when the PR only updates the expected metadata index files, optionally with generated root
COVERAGE.md, and CI is green, then enable auto-merge.
- Close the PR when it contains unrelated changes such as workflow files, Gradle files, generated sources, or test code that clearly did not belong in a bulk version update.
- Close stale PRs with merge conflicts instead of repairing them manually if a newer bulk-update PR will supersede them.
- If CI looks flaky or inconsistent with the diff, ask for or trigger a rerun rather than approving blindly.
- Treat generated-source changes as suspicious in this PR type. They should normally not appear here.
Workflow
Inspect the PR summary.
- Confirm the PR has label
library-bulk-update.
- Read the body and verify it is a bot-generated "Added tested versions" summary.
- Gather files, reviews, issue comments, inline comments, and status checks.
Validate the diff scope first.
- Expected files:
metadata/<group>/<artifact>/index.json, plus the generated root COVERAGE.md when the PR updates tested versions.
- Normal change shape: new values appended to
tested-versions, with formatting-only line movement around the surrounding JSON.
- Be suspicious of changes to
latest, metadata-version, test-version, allowed-packages, requires, or URL fields unless the PR clearly needs them.
- A pure pre-release-to-release promotion produced by
addTestedVersion is allowed when it is limited to the matching library and consists only of:
metadata/<group>/<artifact>/<oldVersion> -> <newVersion>
tests/src/<group>/<artifact>/<oldVersion> -> <newVersion>
stats/<group>/<artifact>/<oldVersion> -> <newVersion>
- matching
index.json changes to metadata-version, tested-versions, and exact-version artifact URLs
- Reject or close the PR if it changes:
.github/workflows/**
ci.json
gradle/**, build.gradle, settings.gradle, gradle.properties
tests/src/** outside a pure matching pre-release promotion rename
stats/** outside a pure matching pre-release promotion rename
- generated Java sources
- any other non-
metadata/**/index.json file, except the generated root COVERAGE.md
Check that the PR body matches the diff.
- Every library listed in the body should correspond to an
index.json diff.
- Added tested versions should match the actual JSON changes.
- A
COVERAGE.md refresh is allowed as generated fallout from the listed library updates; it does not need to be listed as a separate library change.
- There should not be hidden updates outside the listed libraries, except the generated root
COVERAGE.md.
Check CI before deciding.
- Required baseline: JSON validation, metadata tests, and style checks must be green.
- If the PR unexpectedly triggered build-logic or workflow-heavy jobs, that is a sign the diff scope is wrong. A root
COVERAGE.md change alone is not workflow-heavy.
- If checks failed because the PR is contaminated by unrelated changes, close it.
- If checks failed due to likely infra flakiness or a suspicious runner issue, request a rerun or rerun all relevant tests before approval.
Review each changed index.json.
- Confirm the new versions were added under the correct
metadata-version entry.
- Preserve existing ordering conventions; new tested versions are typically appended in version order.
- Do not accept dropped tested versions, entry reshuffling, or unrelated field edits without a clear reason.
- For pre-release promotions, confirm
metadata-version moved from the matching pre-release to the corresponding full release, old pre-release values were removed from tested-versions, and any test-version references were updated only when they pointed at the promoted test directory.
Apply URL verification when URL fields changed.
- This is higher scrutiny than a normal bulk-update PR.
- Review
source-code-url, test-code-url, documentation-url, and repository-url with the same rules used by PopulateArtifactURLs.
- For pre-release promotions, verify the
source-code-url, test-code-url, and documentation-url values were rendered from the old version to the promoted metadata-version or otherwise corrected to an exact-version URL. Do not accept stale URLs that still point at the old pre-release.
If the PR is clean and approved, enable auto-merge.
- In this repository, approved automation PRs are expected to auto-merge after approval.
- Do this after the approval is submitted, and only when the PR is otherwise mergeable.
URL Verification Rules
If any URL field changed, verify all of the following:
repository-url is the canonical repository root URL.
repository-url must not include a versioned tree path such as /tree/<tag>.
source-code-url, test-code-url, and documentation-url must point to the exact library version in the changed entry.
- Do not accept unversioned docs,
latest, current, or branch-based docs unless there is no versioned source and the PR clearly justifies it.
- Prefer Maven
-sources.jar, -test-sources.jar, and -javadoc.jar when they exist and are valid.
- If a Maven
-sources.jar or -test-sources.jar is used, verify it contains real source files such as .java, .kt, .scala, or .groovy, not only metadata or license files.
- If the URL points to a repository tree or archive instead of Maven, verify that it resolves to real source or test files for the exact version tag.
- If a candidate source or test URL cannot be verified with confidence, it should not be approved as-is.
- For pre-release promotions, template rendering is expected: old-version URL tokens may be replaced by the promoted version only when the rendered URL resolves and archive/source checks pass.
Use the logic from tests/tck-build-logic/src/main/groovy/org/graalvm/internal/tck/harness/tasks/PopulateArtifactURLs.java, especially the urlUpdateInstructions and sourceArtifactVerificationInstructions rules.
Decision Rules
Approve when all of these are true:
- The PR is limited to the expected
metadata/**/index.json files, optionally with generated root COVERAGE.md.
- Or the PR is a pure verified pre-release promotion rename limited to the matching
metadata/, tests/src/, stats/, index.json, and optional root COVERAGE.md changes.
- The diff is only tested-version maintenance or clearly justified URL maintenance.
- CI is green, or any rerun confirms green status.
- Any changed URLs were verified against the exact version.
- Auto-merge was enabled after approval when the PR is mergeable.
Close or reject when any of these are true:
- The PR contains unrelated files or generated code.
- The PR is stale, conflicted, or clearly superseded by a newer automation run.
- CI failures point to a real regression or invalid update.
- URL changes cannot be verified confidently.
Ask for rerun or deeper investigation when:
- The diff looks correct but CI failed in a way that smells like infrastructure noise.
- A generated file changed and you need to confirm whether the workflow accidentally regenerated something.
Output Style
Match the historical style:
- Clean PR: approve with no comment or a very short confirmation, then enable auto-merge.
- Contaminated PR: leave a short factual comment explaining why it is being closed or should not be merged.
- Flaky CI: leave a short comment requesting or noting a rerun.
1---2name: review-library-bulk-update3description: Review automated pull requests with the `library-bulk-update` label in graalvm-reachability-metadata. Use when asked to review or triage a PR that bumps tested versions for existing libraries, including approve vs close decisions, CI checks, diff-scope validation, and verification of `source-code-url`, `test-code-url`, `documentation-url`, and `repository-url` changes.4metadata:5 source: https://skillmd.com/skills/oracle/review-library-bulk-update6---78# Review `library-bulk-update` PRs910These PRs are created by the scheduled compatibility workflow and are expected to update supported library versions by changing `metadata/**/index.json`. They may also refresh the generated root `COVERAGE.md` when tested-version changes affect the coverage table.1112## Historical Review Logic1314Previous reviews follow a simple pattern:1516- Approve silently when the PR only updates the expected metadata index files, optionally with generated root `COVERAGE.md`, and CI is green, then enable auto-merge.17- Close the PR when it contains unrelated changes such as workflow files, Gradle files, generated sources, or test code that clearly did not belong in a bulk version update.18- Close stale PRs with merge conflicts instead of repairing them manually if a newer bulk-update PR will supersede them.19- If CI looks flaky or inconsistent with the diff, ask for or trigger a rerun rather than approving blindly.20- Treat generated-source changes as suspicious in this PR type. They should normally not appear here.2122## Workflow23241. Inspect the PR summary.25 - Confirm the PR has label `library-bulk-update`.26 - Read the body and verify it is a bot-generated "Added tested versions" summary.27 - Gather files, reviews, issue comments, inline comments, and status checks.28292. Validate the diff scope first.30 - Expected files: `metadata/<group>/<artifact>/index.json`, plus the generated root `COVERAGE.md` when the PR updates tested versions.31 - Normal change shape: new values appended to `tested-versions`, with formatting-only line movement around the surrounding JSON.32 - Be suspicious of changes to `latest`, `metadata-version`, `test-version`, `allowed-packages`, `requires`, or URL fields unless the PR clearly needs them.33 - A pure pre-release-to-release promotion produced by `addTestedVersion` is allowed when it is limited to the matching library and consists only of:34 - `metadata/<group>/<artifact>/<oldVersion> -> <newVersion>`35 - `tests/src/<group>/<artifact>/<oldVersion> -> <newVersion>`36 - `stats/<group>/<artifact>/<oldVersion> -> <newVersion>`37 - matching `index.json` changes to `metadata-version`, `tested-versions`, and exact-version artifact URLs38 - Reject or close the PR if it changes:39 - `.github/workflows/**`40 - `ci.json`41 - `gradle/**`, `build.gradle`, `settings.gradle`, `gradle.properties`42 - `tests/src/**` outside a pure matching pre-release promotion rename43 - `stats/**` outside a pure matching pre-release promotion rename44 - generated Java sources45 - any other non-`metadata/**/index.json` file, except the generated root `COVERAGE.md`46473. Check that the PR body matches the diff.48 - Every library listed in the body should correspond to an `index.json` diff.49 - Added tested versions should match the actual JSON changes.50 - A `COVERAGE.md` refresh is allowed as generated fallout from the listed library updates; it does not need to be listed as a separate library change.51 - There should not be hidden updates outside the listed libraries, except the generated root `COVERAGE.md`.52534. Check CI before deciding.54 - Required baseline: JSON validation, metadata tests, and style checks must be green.55 - If the PR unexpectedly triggered build-logic or workflow-heavy jobs, that is a sign the diff scope is wrong. A root `COVERAGE.md` change alone is not workflow-heavy.56 - If checks failed because the PR is contaminated by unrelated changes, close it.57 - If checks failed due to likely infra flakiness or a suspicious runner issue, request a rerun or rerun all relevant tests before approval.58595. Review each changed `index.json`.60 - Confirm the new versions were added under the correct `metadata-version` entry.61 - Preserve existing ordering conventions; new tested versions are typically appended in version order.62 - Do not accept dropped tested versions, entry reshuffling, or unrelated field edits without a clear reason.63 - For pre-release promotions, confirm `metadata-version` moved from the matching pre-release to the corresponding full release, old pre-release values were removed from `tested-versions`, and any `test-version` references were updated only when they pointed at the promoted test directory.64656. Apply URL verification when URL fields changed.66 - This is higher scrutiny than a normal bulk-update PR.67 - Review `source-code-url`, `test-code-url`, `documentation-url`, and `repository-url` with the same rules used by `PopulateArtifactURLs`.68 - For pre-release promotions, verify the `source-code-url`, `test-code-url`, and `documentation-url` values were rendered from the old version to the promoted `metadata-version` or otherwise corrected to an exact-version URL. Do not accept stale URLs that still point at the old pre-release.69707. If the PR is clean and approved, enable auto-merge.71 - In this repository, approved automation PRs are expected to auto-merge after approval.72 - Do this after the approval is submitted, and only when the PR is otherwise mergeable.7374## URL Verification Rules7576If any URL field changed, verify all of the following:7778- `repository-url` is the canonical repository root URL.79- `repository-url` must not include a versioned tree path such as `/tree/<tag>`.80- `source-code-url`, `test-code-url`, and `documentation-url` must point to the exact library version in the changed entry.81- Do not accept unversioned docs, `latest`, `current`, or branch-based docs unless there is no versioned source and the PR clearly justifies it.82- Prefer Maven `-sources.jar`, `-test-sources.jar`, and `-javadoc.jar` when they exist and are valid.83- If a Maven `-sources.jar` or `-test-sources.jar` is used, verify it contains real source files such as `.java`, `.kt`, `.scala`, or `.groovy`, not only metadata or license files.84- If the URL points to a repository tree or archive instead of Maven, verify that it resolves to real source or test files for the exact version tag.85- If a candidate source or test URL cannot be verified with confidence, it should not be approved as-is.86- For pre-release promotions, template rendering is expected: old-version URL tokens may be replaced by the promoted version only when the rendered URL resolves and archive/source checks pass.8788Use the logic from `tests/tck-build-logic/src/main/groovy/org/graalvm/internal/tck/harness/tasks/PopulateArtifactURLs.java`, especially the `urlUpdateInstructions` and `sourceArtifactVerificationInstructions` rules.8990## Decision Rules9192Approve when all of these are true:9394- The PR is limited to the expected `metadata/**/index.json` files, optionally with generated root `COVERAGE.md`.95- Or the PR is a pure verified pre-release promotion rename limited to the matching `metadata/`, `tests/src/`, `stats/`, `index.json`, and optional root `COVERAGE.md` changes.96- The diff is only tested-version maintenance or clearly justified URL maintenance.97- CI is green, or any rerun confirms green status.98- Any changed URLs were verified against the exact version.99- Auto-merge was enabled after approval when the PR is mergeable.100101Close or reject when any of these are true:102103- The PR contains unrelated files or generated code.104- The PR is stale, conflicted, or clearly superseded by a newer automation run.105- CI failures point to a real regression or invalid update.106- URL changes cannot be verified confidently.107108Ask for rerun or deeper investigation when:109110- The diff looks correct but CI failed in a way that smells like infrastructure noise.111- A generated file changed and you need to confirm whether the workflow accidentally regenerated something.112113## Output Style114115Match the historical style:116117- Clean PR: approve with no comment or a very short confirmation, then enable auto-merge.118- Contaminated PR: leave a short factual comment explaining why it is being closed or should not be merged.119- Flaky CI: leave a short comment requesting or noting a rerun.