# Review Library Bulk Update > Review automated pull requests with the `library-bulk-update` label in graalvm-reachability-metadata. Use when asked to review or triage a PR that bumps tested versions for existing libraries, including approve vs close decisions, CI checks, diff-scope validation, and verification of `source-code-url`, `test-code-url`, `documentation-url`, and `repository-url` changes. - Skill: `oracle/review-library-bulk-update` (Agent Skill) - Install (CLI): `npx skillmds@latest add oracle/review-library-bulk-update` - Raw SKILL.md: https://api.skillmd.com/api/skills/oracle/review-library-bulk-update/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Docs & Writing - Author: Oracle (https://skillmd.com/u/oracle) - Updated: 2026-09-10 - Page: https://skillmd.com/skills/oracle/review-library-bulk-update --- # Review `library-bulk-update` PRs These PRs are created by the scheduled compatibility workflow and are expected to update supported library versions by changing `metadata/**/index.json`. They may also refresh the generated root `COVERAGE.md` when tested-version changes affect the coverage table. ## Historical Review Logic Previous reviews follow a simple pattern: - Approve silently when the PR only updates the expected metadata index files, optionally with generated root `COVERAGE.md`, and CI is green, then enable auto-merge. - Close the PR when it contains unrelated changes such as workflow files, Gradle files, generated sources, or test code that clearly did not belong in a bulk version update. - Close stale PRs with merge conflicts instead of repairing them manually if a newer bulk-update PR will supersede them. - If CI looks flaky or inconsistent with the diff, ask for or trigger a rerun rather than approving blindly. - Treat generated-source changes as suspicious in this PR type. They should normally not appear here. ## Workflow 1. Inspect the PR summary. - Confirm the PR has label `library-bulk-update`. - Read the body and verify it is a bot-generated "Added tested versions" summary. - Gather files, reviews, issue comments, inline comments, and status checks. 2. Validate the diff scope first. - Expected files: `metadata///index.json`, plus the generated root `COVERAGE.md` when the PR updates tested versions. - Normal change shape: new values appended to `tested-versions`, with formatting-only line movement around the surrounding JSON. - Be suspicious of changes to `latest`, `metadata-version`, `test-version`, `allowed-packages`, `requires`, or URL fields unless the PR clearly needs them. - A pure pre-release-to-release promotion produced by `addTestedVersion` is allowed when it is limited to the matching library and consists only of: - `metadata/// -> ` - `tests/src/// -> ` - `stats/// -> ` - matching `index.json` changes to `metadata-version`, `tested-versions`, and exact-version artifact URLs - Reject or close the PR if it changes: - `.github/workflows/**` - `ci.json` - `gradle/**`, `build.gradle`, `settings.gradle`, `gradle.properties` - `tests/src/**` outside a pure matching pre-release promotion rename - `stats/**` outside a pure matching pre-release promotion rename - generated Java sources - any other non-`metadata/**/index.json` file, except the generated root `COVERAGE.md` 3. Check that the PR body matches the diff. - Every library listed in the body should correspond to an `index.json` diff. - Added tested versions should match the actual JSON changes. - A `COVERAGE.md` refresh is allowed as generated fallout from the listed library updates; it does not need to be listed as a separate library change. - There should not be hidden updates outside the listed libraries, except the generated root `COVERAGE.md`. 4. Check CI before deciding. - Required baseline: JSON validation, metadata tests, and style checks must be green. - If the PR unexpectedly triggered build-logic or workflow-heavy jobs, that is a sign the diff scope is wrong. A root `COVERAGE.md` change alone is not workflow-heavy. - If checks failed because the PR is contaminated by unrelated changes, close it. - If checks failed due to likely infra flakiness or a suspicious runner issue, request a rerun or rerun all relevant tests before approval. 5. Review each changed `index.json`. - Confirm the new versions were added under the correct `metadata-version` entry. - Preserve existing ordering conventions; new tested versions are typically appended in version order. - Do not accept dropped tested versions, entry reshuffling, or unrelated field edits without a clear reason. - For pre-release promotions, confirm `metadata-version` moved from the matching pre-release to the corresponding full release, old pre-release values were removed from `tested-versions`, and any `test-version` references were updated only when they pointed at the promoted test directory. 6. Apply URL verification when URL fields changed. - This is higher scrutiny than a normal bulk-update PR. - Review `source-code-url`, `test-code-url`, `documentation-url`, and `repository-url` with the same rules used by `PopulateArtifactURLs`. - For pre-release promotions, verify the `source-code-url`, `test-code-url`, and `documentation-url` values were rendered from the old version to the promoted `metadata-version` or otherwise corrected to an exact-version URL. Do not accept stale URLs that still point at the old pre-release. 7. If the PR is clean and approved, enable auto-merge. - In this repository, approved automation PRs are expected to auto-merge after approval. - Do this after the approval is submitted, and only when the PR is otherwise mergeable. ## URL Verification Rules If any URL field changed, verify all of the following: - `repository-url` is the canonical repository root URL. - `repository-url` must not include a versioned tree path such as `/tree/`. - `source-code-url`, `test-code-url`, and `documentation-url` must point to the exact library version in the changed entry. - Do not accept unversioned docs, `latest`, `current`, or branch-based docs unless there is no versioned source and the PR clearly justifies it. - Prefer Maven `-sources.jar`, `-test-sources.jar`, and `-javadoc.jar` when they exist and are valid. - If a Maven `-sources.jar` or `-test-sources.jar` is used, verify it contains real source files such as `.java`, `.kt`, `.scala`, or `.groovy`, not only metadata or license files. - If the URL points to a repository tree or archive instead of Maven, verify that it resolves to real source or test files for the exact version tag. - If a candidate source or test URL cannot be verified with confidence, it should not be approved as-is. - For pre-release promotions, template rendering is expected: old-version URL tokens may be replaced by the promoted version only when the rendered URL resolves and archive/source checks pass. Use the logic from `tests/tck-build-logic/src/main/groovy/org/graalvm/internal/tck/harness/tasks/PopulateArtifactURLs.java`, especially the `urlUpdateInstructions` and `sourceArtifactVerificationInstructions` rules. ## Decision Rules Approve when all of these are true: - The PR is limited to the expected `metadata/**/index.json` files, optionally with generated root `COVERAGE.md`. - Or the PR is a pure verified pre-release promotion rename limited to the matching `metadata/`, `tests/src/`, `stats/`, `index.json`, and optional root `COVERAGE.md` changes. - The diff is only tested-version maintenance or clearly justified URL maintenance. - CI is green, or any rerun confirms green status. - Any changed URLs were verified against the exact version. - Auto-merge was enabled after approval when the PR is mergeable. Close or reject when any of these are true: - The PR contains unrelated files or generated code. - The PR is stale, conflicted, or clearly superseded by a newer automation run. - CI failures point to a real regression or invalid update. - URL changes cannot be verified confidently. Ask for rerun or deeper investigation when: - The diff looks correct but CI failed in a way that smells like infrastructure noise. - A generated file changed and you need to confirm whether the workflow accidentally regenerated something. ## Output Style Match the historical style: - Clean PR: approve with no comment or a very short confirmation, then enable auto-merge. - Contaminated PR: leave a short factual comment explaining why it is being closed or should not be merged. - Flaky CI: leave a short comment requesting or noting a rerun.