Always find newest Python 3 executable on machine

[dougbu]

our current code uses a few different approaches to find the "correct" Python 3 executable on a machine. this is problematic for a few reasons e.g.,

• support for older versions of Python is quickly dropped in newer pip and cryptography releases (probably more)
• Python releases themselves incorporate security fixes
• only Python 3.8 and newer are apparently in support (Python 3.8 will go EOL 14 Oct 2024)
• when multiple Python 3 versions are available on a machine and two approaches choose too different versions, pip and other installed packages including our wheels may not be usable. that is, package installations are specific to the chosen Python version
• OS vendors seem to be doing a good job keeping up w/ Python.org

for example, some of our Linux code always looks in /usr/bin for the executable, but our custom OSX installation places the executable in /usr/local/bin. we also special case python3.6 in a few places, but not everywhere

note this issue is not about using e.g. python3.9-dev where that's available. we might want to investigate the exact version python3-dev (or similar) corresponds to these days. but that's separate from where we find Python 3.

I have most of the work done in an existing dotnet-helix-machines PR. will be separating that out for consideration on its own soon.

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Updated our Python 3 discovery scripts and code to always find the newest executable on a machine. This improves consistency and security of our features, as well as enabling more rapid Python package version bumps going forward.

[dougbu]

one small caveat here: long-lived (i.e., on premise) machines may get new Python 3 features during OS or artefact updates (see #379, especially mid-term thoughts about keeping artefacts up to date. that could mean an OS update which moves from e.g., Python 3.8 to 3.9 loses information about installed packages. fortunately, this is extremely unlikely given the care vendors put into their package upgrade procedures

[dougbu]

I'd like to emphasize and update the most important benefits of this PR:

1. when multiple Python 3 versions are available on a machine and two approaches choose too different versions, pip and other installed packages including our wheels may not be usable. that is, package installations are specific to the chosen Python version
2. fixing this allows incremental updates to the artefacts on our on-prem machines. a new Python major or minor version can now be added to an image w/o changing the python3 symbolic link. OS packages do that automatically in all cases I've seen. (it's like they invoke make altinstall instead of make install; without the whole build part of course.) see also Get helix-scripts/ up to date on on-premise machines #377

The points about EOLs and security are actually more related to #465 though the Python 3.11 installation on vmimage: macos-* in our primary pipeline blurs the line.

/fyi @garath

[dougbu]

blocked by work on #1371 — that must be merged first

[dougbu]

unblocked now that #1371 fix is in

[dougbu]

Resolved w/ !35549