refactor(pypi): translate wheel METADATA parsing to starlark #2629
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Mar 23, 2025
github-merge-queue bot
pushed a commit
that referenced
this pull request
Mar 30, 2025
This implements the PEP508 compliant marker evaluation in starlark and removes the need for the Python interpreter when evaluating requirements files passed to `pip.parse`. This makes the evaluation faster and allows us to fix a few known issues (#2690). In the future the intent is to move the `METADATA` parsing to pure starlark so that the `RequiresDist` could be parsed in starlark at the macro evaluation or analysis phases. This should make it possible to more easily solve the design problem that more and more things need to be passed to `whl_library` as args to have a robust dependency parsing: * #2319 needs the full Python version to have correct cross-platform compatible `METADATA` parsing and passing it to `Python` and back makes it difficult/annoying to implement. * Parsing the `METADATA` file requires the precise list of target platform or the list of available packages in the `requirements.txt`. This means that without it we cannot trim the dependency tree in the `whl_library`. Doing this at macro loading phase allows us to depend on `.bzl` files in the `hub_repository` and more effectively pass information. I can remotely see that this could become useful in `py_wheel` or an building wheels from sdists as the environment markers may be present in various source metadata as well. What is more `uv.lock` file has the env markers as part of the lock file information, so this might be useful there. Work towards #2423 Work towards #260 Split from #2629
This reverts commit 1de8269.
This reverts commit 7f99cb7.
Since toolchain matching is done by matching the first target that matches target settings, the `minor_mapping` config setting is special, because e.g. all `3.11.X` toolchains match the `python_version = "3.11"` setting. This just reshuffles the list so that we have toolchains that are in the `minor_mapping` before the rest. Fixes bazel-contrib#2685
|
For having unit tests working we need to first resolve #2735. |
aignas
changed the title
aignas
force-pushed
the
feat/starlark-env-marker-and-metadata-parsing
branch
from
2d6d4c5 to
c4b08ad
Compare
aignas
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR starts using the newly introduced (#2692) PEP508 compliant
requirement marker parser in starlark and moves the dependency
generation from the Python language (
whl_installer) to the Starlarkin the
whl_libraryrepository rule.This PR is (almost) a pure refactor where no bugs are fixed, but this is
foundational work that also adds notes on how things will be moved
to macros (i.e. analysis phase) so that we can fix a few long standing
bugs and prepare for stabilizing the
experimental_index_url(#260).I have migrated all of the unit tests from Python to starlark.
Non refactoring things:
(())- a test has been added.Work towards #260, #2319, #2241
Fixes #2423