Third Party Package Patches

Third-Party Package Patches

Use this skill when creating or updating compatibility patches for packages installed by pip on GraalPy.

Key Files

• Source preparation: scripts/get_pypi_source.py
• Patch metadata: graalpython/lib-graalpython/patches/metadata.toml
• Patch directory: graalpython/lib-graalpython/patches/
• Cargo autopatcher: graalpython/lib-graalpython/modules/autopatch_cargo.py
• Crate patch metadata: graalpython/lib-graalpython/patches/crates/metadata.toml
• Metadata verifier: mx.graalpython/verify_patches.py

Workflow

1. Identify the package name and target version. Use the normalized package key used by PyPI/pip: lowercase with runs of -, _, and . normalized to -.

2. Prepare source with the repo helper:

python scripts/get_pypi_source.py package==version

The script prints Prepared source at: .... Use that directory as the working tree. It is already a temporary git repository with an initial commit, and it has already been processed by autopatch_capi.py and autopatch_cargo.py.

3. Inspect graalpython/lib-graalpython/patches/metadata.toml for existing [[package.rules]] entries.

• If a matching patch exists for the requested version, apply it first.
• If only a nearby version has a patch, try applying that patch and rebase it carefully onto the prepared source.
• Honor subdir when present: pip applies sdist patches from that subdirectory. Apply and generate the patch from the same directory layout that the metadata rule will use.
• Honor dist-type when choosing whether the patch is for sdist, wheel, or both.

4. Apply existing patches using the same semantics as pip where practical:

patch -f -d /tmp/package-version-... -p1 -i /path/to/graalpython/lib-graalpython/patches/existing.patch

If the rule has subdir = 'src', use -d /tmp/package-version-.../src. Resolve rejects by editing source files in the temporary repository, then remove any .rej/.orig files after checking them. Search for conflict markers and rejects before staging.

5. Make the GraalPy compatibility changes in the prepared source. Keep the patch minimal and package-focused.

6. Stage the desired changes in the temporary source repository:

git add -A
git diff --cached

Review the staged diff. Do not include generated caches, build outputs, rejected patch files, or unrelated churn.

7. Create or refresh the patch file only from the staged git diff:

git diff --cached > /path/to/graalpython/lib-graalpython/patches/package-version.patch

Guardrail: do not hand-edit patch files. If the patch output is wrong, fix the temporary source tree, adjust staging, and regenerate with git diff --cached.

8. Update metadata.toml if needed.

• Add a new [[package.rules]] entry when no suitable one exists.
• Keep existing precedent for rule ordering, patch names, comments, install-priority, dist-type, and subdir.
• Every rule with patch = ... needs license = ....
• When adding a new patch entry, confirm the license from PyPI metadata, preferably the JSON API for the exact release or current package metadata. Use an SPDX identifier accepted by mx.graalpython/verify_patches.py.
• If upstream publishes no suitable PyPI source artifact, add [[package.add-sources]] with the exact version and release tarball URL, then rerun scripts/get_pypi_source.py.

9. Choose the version range deliberately.

• Prefer one patch over many when the same patch applies cleanly and the underlying package layout/API is stable.
• Test every version covered by a widened range, including lower and upper boundary releases.
• Small, robust patches may use an open-ended range when they are unlikely to break in future versions.
• If newer versions no longer need a patch, add a no-patch rule with a note rather than leaving users pointed at stale patched versions.

10. Validate patch application for the covered versions.

• For each version in the rule range that you claim to support, prepare a fresh source tree with scripts/get_pypi_source.py.
• Apply the generated patch with patch -f -p1 from the root or subdir exactly as the metadata rule requires.
• Check for nonzero exit status, .rej files, .orig files, unexpected unstaged files, and conflict markers.

11. Run the repository verifier before finishing:

python mx.graalpython/verify_patches.py graalpython/lib-graalpython/patches

12. If you were asked to build or test the patched package, you need to rebuild GraalPy with mx python-jvm to pick up the changes. Create a venv with mx python -m venv venv_name and use it for building and testing.

Rust Crate Patches

Use autopatch_cargo.py when the incompatibility is in a transitive crates.io dependency rather than the Python package source. It finds matching crates in Cargo.lock, copies only those crates into the source tree, applies registered patches, and adds Cargo path overrides.

1. Add rules to graalpython/lib-graalpython/patches/crates/metadata.toml. Use the crate package name as the table key and specify version, patch, and license for every rule.
2. Generate crate patch files from a clean checkout of the published crate. Patch paths must be relative to the crate root. Do not hand-edit generated patch files.
3. Run the tool directly when validating a prepared source tree:

python graalpython/lib-graalpython/modules/autopatch_cargo.py /path/to/source

4. Test every crate release covered by the version range, then build representative Python packages that resolve those releases.
5. Use the Python package rule's autopatch = false only when both autopatch_capi and autopatch_cargo must be disabled for that package version.

Metadata Reference

Rule keys accepted by the verifier are:

• version
• patch
• license
• subdir
• dist-type
• install-priority
• note
• autopatch

Allowed dist-type values are wheel and sdist.

Allowed license identifiers are maintained in mx.graalpython/verify_patches.py. If PyPI metadata is ambiguous, inspect the source distribution license files and report the ambiguity instead of guessing.

Reporting

When done, report:

• package and version(s) tested
• patch file created or updated
• metadata rule added or changed
• PyPI license value used
• verify_patches.py result