Autogenerate environment.yml file from pyproject.toml

[drammock]

closes #12903

Easiest to review a few specific commits:

• a3f2a0e adds the hook
• 1a0d7b9 does nothing except sort environment.yml entries so that subsequent hook-induced changes to that file are easier to see
• ef61f28 shows the actual changes caused by the hook on first run (plus a fake "aaaaaaa" dep added to trigger the hook, which is removed in subsequent commit 4e99502)

Note that this does not address #12903 (comment) --- I figured easier to get this one in / tested and then deal later with cross-repo syncing.

[drammock]

windows-latest mamba job will continue to fail until the fix for mamba-org/mamba#3467 has been released.

[larsoner]

It occurred to me that any autogenerated file should have a comment so I just pushed a tiny commit to add this to the top of the env file:

# THIS FILE IS AUTO-GENERATED BY tools/hooks/update_environment_file.py AND WILL BE OVERWRITTEN

similar to what we have in doc/sphinxext/credit_tools.py

[larsoner]

Ughhhh looks like I was wrong about openblas, will push a commit

[larsoner]

I am a bit mystified by both the pip-pre failure and the conda one. Looking at a diff of the sorted main YML and the one in this PR I don't see anything that should break linalg results as seems to be happening in the conda install now.

[drammock]

I am a bit mystified by both the pip-pre failure and the conda one. Looking at a diff of the sorted main YML and the one in this PR I don't see anything that should break linalg results as seems to be happening in the conda install now.

I'm scratching my head too 😕

[drammock]

the numpy config on windows-latest mamba is:

 Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: C:/Users/runneradmin/micromamba/envs/mne/Library/include
    lib directory: C:/Users/runneradmin/micromamba/envs/mne/Library/lib
    name: blas
    openblas configuration: unknown
    pc file directory: D:\bld\numpy_1707225570061\_h_env\Library\lib\pkgconfig
    version: 3.9.0
  lapack:
    detection method: internal
    found: true
    include directory: unknown
    lib directory: unknown
    name: dep2598594164480
    openblas configuration: unknown
    pc file directory: unknown
    version: 1.26.4

The line blas: openblas configuration: unknown is different on my local machine, which has USE_64BITINT=1 DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= HASWELL MAX_THREADS=2. The version string also looks fishy (I see 0.3.23.dev locally, CI has 3.9.0?? AFAIK openblas versions are all still 0.x)

[drammock]

comparing that to the windows pip-pre job:

Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: C:/Users/runneradmin/AppData/Local/Temp/cibw-run-rk8_4lij/cp312-win_amd64/build/venv/Lib/site-packages/scipy_openblas64/include
    lib directory: C:/Users/runneradmin/AppData/Local/Temp/cibw-run-rk8_4lij/cp312-win_amd64/build/venv/Lib/site-packages/scipy_openblas64/lib
    name: scipy-openblas
    openblas configuration: OpenBLAS 0.3.28  USE64BITINT DYNAMIC_ARCH NO_AFFINITY
      Haswell MAX_THREADS=24
    pc file directory: D:/a/numpy/numpy/.openblas
    version: 0.3.28
  lapack:
    detection method: pkgconfig
    found: true
    include directory: C:/Users/runneradmin/AppData/Local/Temp/cibw-run-rk8_4lij/cp312-win_amd64/build/venv/Lib/site-packages/scipy_openblas64/include
    lib directory: C:/Users/runneradmin/AppData/Local/Temp/cibw-run-rk8_4lij/cp312-win_amd64/build/venv/Lib/site-packages/scipy_openblas64/lib
    name: scipy-openblas
    openblas configuration: OpenBLAS 0.3.28  USE64BITINT DYNAMIC_ARCH NO_AFFINITY
      Haswell MAX_THREADS=24
    pc file directory: D:/a/numpy/numpy/.openblas
    version: 0.3.28

here we see a reasonable version number, known configuration strings, and a different name ("scipy-openblas" instead of "blas")

[drammock]

ok, yep: windows-latest mamba is getting an MKL version of numpy for some reason:

 + numpy                1.26.4 (MKL 2024.2.2-Product with 1 thread)

[drammock]

disabling auto-merge because I'm not certain if 69a790d will actually be necessary anymore.

[drammock]

the corresponding branch in github_actions_dependencies.sh uses test_extra too, and it's needed for utils/tests/test_config.py (I think previously it was passing on windows by chance)

[larsoner]

Thanks @drammock !

[drammock]

@larsoner do you have any idea why we were suddenly getting MKL? I never did track down a cause.

[larsoner]

No idea. Could be unfortunate timing with some conda change. Or something in the solver changing some logic. Or a new package. But I like the nomkl solution!