feat: add linux aarch64 wheel #2103
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mayeut
force-pushed
the
abi3-linux-aarch64
branch
3 times, most recently
from
ddc8969 to
73a80da
Compare
mayeut
force-pushed
the
abi3-linux-aarch64
branch
from
73a80da to
a99ad94
Compare
|
@mayeut is the failing test a flaky one which can be re-run, or does this PR need another fix before @giampaolo can review? awesome work btw 🙏 oh and I think |
mayeut
force-pushed
the
abi3-linux-aarch64
branch
from
a99ad94 to
b8c1753
Compare
It was a flaky test. There was also another fix needed since the As mentioned in #2070 (comment),
This is intentional. It won't fix those issues which are related to building from sources. Of course, people are much less likely to hit those since they would probably get a pre-built wheel unless enforcing a build from source. Hence |
mayeut
force-pushed
the
abi3-linux-aarch64
branch
from
b8c1753 to
80d814c
Compare
|
For people eager to test these awesome wheels in the meantime: Using $ export PIP_FIND_LINKS='https://github.com/ddelange/psutil/releases/expanded_assets/release-5.9.1' # space separated links
$ pip install --no-cache-dir psutil==5.9.1
Collecting psutil~=5.9
Downloading https://github.com/ddelange/psutil/releases/download/release-5.9.1/psutil-5.9.1-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (271 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 271.3/271.3 KB 22.2 MB/s eta 0:00:00As this is dynamic, definitely unsafe for production. I recommend forking to have some immutability. |
mayeut
force-pushed
the
abi3-linux-aarch64
branch
from
80d814c to
997ab3c
Compare
mayeut
force-pushed
the
abi3-linux-aarch64
branch
from
997ab3c to
e99529c
Compare
|
@giampaolo could you please review this awesome contribution? |
mayeut
force-pushed
the
abi3-linux-aarch64
branch
2 times, most recently
from
19490c3 to
ba013d8
Compare
This comment was marked as outdated.
This comment was marked as outdated.
mayeut
force-pushed
the
abi3-linux-aarch64
branch
2 times, most recently
from
4e4d7bc to
ca86104
Compare
mayeut
force-pushed
the
abi3-linux-aarch64
branch
from
ca86104 to
5877dce
Compare
|
Whats the progress for this? Seems my docker image won't compile even after installing |
|
as long as this PR is not merged and released, pip will get the source distribution and not the arm64 wheel that this PR will add to the build pipeline. hotfix in my previous comment above |
alas, this didn't seem to work: I guess there is nothing to do but wait? |
|
for completeness this is what I ran: |
|
nice article explaining issue: https://pythonspeed.com/articles/docker-build-problems-mac/ |
mayeut
force-pushed
the
abi3-linux-aarch64
branch
from
de29694 to
32934ed
Compare
This commit updates the build workflow to use the latest cibuildwheel as a GitHub Action. cibuildwheel configuration is now in its own file (as there's no `pyproject.toml` yet) Signed-off-by: mayeut <[email protected]>
Signed-off-by: mayeut <[email protected]>
`macos-10.15` runner is deprecated. Signed-off-by: mayeut <[email protected]>
Signed-off-by: mayeut <[email protected]>
Signed-off-by: mayeut <[email protected]>
Signed-off-by: mayeut <[email protected]>
mayeut
force-pushed
the
abi3-linux-aarch64
branch
from
32934ed to
0daf018
Compare
|
@giampaolo can you please review? |
|
Hello. |
|
Hi @giampaolo 👋 Appreciate you checking in. I don't know @mayeut, but I have seen him putting a lot of time and effort into this PR over the past 4 (!) months. To me your comment is disappointing, I can imagine it similar for @mayeut.
I think the features are nicely separated, the biggest part of the diff comes in #2102: switching to limited ABI (only two tiny changes in
With 60+ million downloads per month, this package comes in very high in the 'hall of fame'. I completely understand your effort considerations, and wouldn't want you to get overloaded due to this change. I think, however, that for such an essential package there are enough professional developers out there for whom it would not be a lot of work to handle some aarch64 bugfixes in their spare/allocated OSS time. So if you don't have time for such fixes, which I understand, there will be a contributor sooner or later that will take it over (this effort from @mayeut as a beautiful example). I'm afraid that waving off aarch64 so quickly isn't that easy in reality: as the PR description points out, aarch64 has already surpassed windows in terms of downloads for this package. Oracle, Amazon, Google, and Microsoft are all offering aarch64 cloud instances at an undeniable price point compared to amd/intel, so the demand will undoubtedly only grow. As for me, I only have experience with Amazon EC2, and know that an install from source on their Graviton processors will fail without further config when there are no wheels available. That's something that many devs will run into during their regular day at work. There is a lot of value in this PR. Please consider merging and releasing, for Python community's sake 🙏 |
Unfortunately that's not how it works, mostly because binaries are directly related to releases, and those are up to me. It’s not about reviewing and merging one PR. It’s about the additional and continuous work required after a PR like this gets merged. CI builds / binaries distribution is that part of the project infrastructure which requires a constant maintenance over time, because OS and python versions constantly change, and so does GitHub infrastructure / config matrix, which keeps breaking with time. Once wheels for a new platform are offered, you basically are forced to keep supporting them. I may change my mind in the future, but this is not sustainable for me at the moment. I may have some ideas on how to try make this sustainable in the long run, but I need time to think about it. |
|
It saddens me to see this PR get closed like that, @giampaolo. Let me try to advocate one more time with regards to your last comment: because CI already tests the binaries right after creating them, and because green CI is required to merge PRs, I don't think you'll run into the drift issues you describe as long as you make a release relatively soon after the last PR went in. However, I can relate to your prudence as person responsible for releases. Maybe CI on a cron job for In the meantime, the show must go on. To offer the community an alternative to this dead end, and to not let @mayeut's efforts be in vain: my fork includes the commits from this PR on export PIP_FIND_LINKS=https://ddelange.github.io/psutil/
pip install ...
# OR
pip install ... --find-links=https://ddelange.github.io/psutil/ |
|
@giampaolo, thanks for your detailed answers. While I'm a bit disappointed, I do appreciate the position you'd be in w.r.t. to maintaining if it got merged in.
I'll clean the other PRs with that in mind. |
|
I trued @mayeut fork and can't install it on NVIDIA Jetson Nano B01 with Python3.8.0 (the OS came with Python2 and Python3.6.9 but anomalib require Python3.8, so I use Python3.8.0) What i've did: |
|
Thanks for the info. I will definitely try python3.8-dev. Previously I install python3.8. I use I will use |
1 task
|
@giampaolo, I'll try to give you an update before end of next week including some comments on the new PR if you want some. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
fixes [Linux] Publish armv7l & aarch64 wheels #1966
fixes [Linux][aarch64] Wheel support for aarch64 Linux #1972
mitigates [AWS Graviton2] graviton gcc flags break pip install #2090
Description
Build Linux aarch64 wheels.
Linux aarch64 downloads now represents around 8% of downloads. Having binary wheels will help users to install psutil as can be seen in multiple issues.