Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix squished zip key ordering #1815

Merged
merged 1 commit into from
Jan 16, 2024

Conversation

mbargull
Copy link
Member

Checklist

  • Added a news entry

Fixes gh-1459
Fixes the conda-smithy part affecting gh-1782 (that issue needs further adjustments to the recipe and/or conda-build).

gh-1459 is an old issue such that the initial discussion refers to old code.
A recent analysis regarding current code is at #1459 (comment) .
This PR adds the suggested change from #1459 (comment) .

With that change the issue from conda-forge/python-feedstock#656 (comment) has not been reproducible for me (for a selection of PYTHONHASHSEED values).
Similarly the issue from conda-forge/pytensor-suite-feedstock#54 reported here at gh-1782 does not reproduce for me (when fixing top-level py selector without {{ python }} usage in that recipe).

@jakirkham
Copy link
Member

Is there a test we could include? Admittedly this may be tricky

@mbargull mbargull marked this pull request as draft December 20, 2023 22:45
@beckermr
Copy link
Member

@mbargull said on the call just now that the code above is not correct and needs changes

Values from top level keys with possibly (as in hash seed-dependent)
different ordering overwrote some of the determined used variants.
If values of related zipped keys were left untouched, this led to
mismatched ordering of zip_keys entries.

ref:
- conda-forge#1459 (comment)

Signed-off-by: Marcel Bargull <[email protected]>
@mbargull mbargull force-pushed the fix-squished-zip_key-ordering branch from 22fc34c to 85482fd Compare January 12, 2024 19:00
@mbargull mbargull marked this pull request as ready for review January 12, 2024 19:02
Copy link
Member

@beckermr beckermr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to merge assuming you've tested that this fixes the issue on a feedstock.

@beckermr
Copy link
Member

A test here is going to be pretty hard to make.

Comment on lines +398 to +403
# NOTE: "pin_run_as_build" (dict of dict) order is not handled.
# "tuple" below only used for uniform list/tuple/str comparison.
v_set = set(map(tuple, v))
v = type(input_variant)(
v_i for v_i in input_variant if tuple(v_i) in v_set
)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opted to no handle pin_run_as_build since its order should not matter, i.e., we'd had to do special handling for that case here with no further benefit.
I added at least a note so we are aware that it's intentionally omitted.

The variant entries should be of type str, so a straight v_set = set(v) would be possible.
We do, however, also have zip_keys which is a list of list/tuple.
To handle potential tuple-to-list comparisons (and to get hashable types for list entries) this just casts everything to tuples.

Feel free to amend any of this in case you'd want more explicit handling.

@mbargull
Copy link
Member Author

Happy to merge assuming you've tested that this fixes the issue on a feedstock.

Tried this on python-feedstock with some PYTHONHASHSEED values that make it break without these changes.

A test here is going to be pretty hard to make.

Yeah, at least we have https://github.com/conda-forge/conda-smithy/blob/v3.30.4/tests/test_cli.py#L90 (which was actually tripped with the first pass I had with this).

@beckermr
Copy link
Member

I'll wait a day or so and then merge if things are looking good. Thank you!

@mbargull
Copy link
Member Author

I'll wait a day or so and then merge if things are looking good. Thank you!

Makes sense. Thanks for reviewing!

@beckermr beckermr merged commit f1456a3 into conda-forge:main Jan 16, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

docker_image: cos7-cuda:10.2 chosen over cos7-x86_64 for os_version: { linux_64: cos7 }
3 participants