Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

arm64 runner improvements #101

Open
4 tasks
jeremyd2019 opened this issue Dec 9, 2024 · 7 comments
Open
4 tasks

arm64 runner improvements #101

jeremyd2019 opened this issue Dec 9, 2024 · 7 comments

Comments

@jeremyd2019
Copy link
Member

  • use runner groups for self-hosted runners #93
  • Switch to new ARM64 Git for Windows
    • It's still beta, but would probably work ok, and we can get it some exercise.
  • Switch to ARM64 Python
    • just remove the architecture: x64. I tested, but there are not yet binary wheels for win-arm64, so installing requirements fails for lack of visual studio. I'm not interested in installing visual studio just for that.
  • Update to Windows 11 24H2
    • I did some preliminary work (and random commentary) on that in Microsoft improvements for ARM64 in 24H2 jeremyd2019/winautoconfig#5, but I haven't been offered the update yet on any system (though I didn't opt in for latest updates). I'd like to wait until the host gets the update at least. Based on my previous feelings on 'major' Windows upgrades, I'd normally wait a few months for the bugs to be shaken out of the new version.
@lazka
Copy link
Member

lazka commented Dec 9, 2024

* [ ]  Switch to ARM64 Python

None of the packages affected provide arm64 binary wheels. They are likely all waiting for GitHub hosted runners (for example pyca/cryptography#11199) as Python packaging does not support cross compiling (except hackily with setuptools which is on the way out). So I don't expect much progress there. The timeline for those runners was recently moved/adjusted to Q1 2025 (github/roadmap#970)

@jeremyd2019
Copy link
Member Author

jeremyd2019 commented Dec 29, 2024

I've added pre and post job scripts/hooks on my ARM64 runners: jeremyd2019/winautoconfig@202e943...3419a78

@lazka
Copy link
Member

lazka commented Jan 10, 2025

I've added pre and post job scripts/hooks on my ARM64 runners: jeremyd2019/[email protected]

Does that mean we can cancel stuck jobs now?

edit: I tried it, seems to have worked (or it was in a good state): https://github.com/msys2-arm/msys2-autobuild/actions/runs/12702640749/job/35409213279

oh, now it's stuck again in make_tree_writable

@jeremyd2019
Copy link
Member Author

jeremyd2019 commented Jan 10, 2025

Yes, it now should be safer to cancel jobs. Not completely safe, but safer. The fact that it was not building a package means it probably would have been relatively safe before.

oh, now it's stuck again in make_tree_writable

That's git-for-windows/git#5320. Local rmdir /s C:\M\B unstuck it

@jeremyd2019
Copy link
Member Author

jeremyd2019 commented Jan 10, 2025

maybe make_tree_writable needs to detect symlinks/directory junctions and not follow them also? Just thinking out loud

@lazka
Copy link
Member

lazka commented Jan 10, 2025

huh, good point, python/cpython#67596

lazka added a commit to lazka/msys2-autobuild that referenced this issue Jan 10, 2025
As found out here, os.walk() by default follows junctions, which we don't
want and can even lead to loops:
msys2#101 (comment)

Integrate the workaround mentioned in the CPython bug report:
python/cpython#67596 (comment)
Since this is Python 3.12+ only and we still support 3.10 make
it optional though.

This also adds tests, which uncovered some other minor issues:
It was not chmoding top-down, which meant that os.walk would
skip things if there were no read permissions. So chmod before
os.walk() lists the dir.
@lazka
Copy link
Member

lazka commented Jan 10, 2025

I've created #107

lazka added a commit that referenced this issue Jan 10, 2025
As found out here, os.walk() by default follows junctions, which we don't
want and can even lead to loops:
#101 (comment)

Integrate the workaround mentioned in the CPython bug report:
python/cpython#67596 (comment)
Since this is Python 3.12+ only and we still support 3.10 make
it optional though.

This also adds tests, which uncovered some other minor issues:
It was not chmoding top-down, which meant that os.walk would
skip things if there were no read permissions. So chmod before
os.walk() lists the dir.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants