Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: make dependencies versions more reliable #616

Merged
merged 18 commits into from
Jul 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion .github/actions/install-python-and-package/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,19 @@ runs:
activate-environment: deeprank2
environment-file: env/deeprank2.yml
use-mamba: true
if: ${{ inputs.pkg-installation-type != 'frozen' }}

- name: Setup miniconda with frozen dependencies
uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
miniforge-variant: Mambaforge
channels: conda-forge
python-version: ${{ inputs.python-version }}
activate-environment: deeprank2
environment-file: env/deeprank2_frozen.yml
use-mamba: true
if: ${{ inputs.pkg-installation-type == 'frozen' }}

- run: |
conda --version
Expand All @@ -53,7 +66,7 @@ runs:

- name: Install the GitHub repository version of the package
shell: bash -l {0}
if: ${{ inputs.pkg-installation-type == 'repository' }}
if: ${{ inputs.pkg-installation-type == 'repository' || inputs.pkg-installation-type == 'frozen' }}
run: |
conda activate deeprank2
pip install .'[${{ inputs.extras-require }}]'
Expand Down
60 changes: 60 additions & 0 deletions .github/workflows/build-repo-frozen-env.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: build (repository package) using the frozen environment

on:
push:
paths-ignore:
# specific folder locations
- ".vscode/**"
- "docs/**"
# filetypes
- "**.md"
- "**.rst"
- "**.ipynb"
- "**.cff"
- "**.png"
branches:
- main
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
paths-ignore:
# specific folder locations
- ".vscode/**"
- "docs/**"
# filetypes
- "**.md"
- "**.rst"
- "**.ipynb"
- "**.cff"
- "**.png"

jobs:
build:
if: github.event.pull_request.draft == false
name: Build for (${{ matrix.python-version }}, ${{ matrix.os }})
runs-on: ${{ matrix.os }}

strategy:
fail-fast: false
matrix:
os: ["ubuntu-latest"]
python-version: ["3.10"] # ["3.10", "3.11"]

# https://github.com/marketplace/actions/setup-miniconda#use-a-default-shell
defaults:
run:
shell: bash -l {0}

steps:
- uses: actions/checkout@v3

- uses: ./.github/actions/install-python-and-package
with:
python-version: ${{ matrix.python-version }}
extras-require: test, publishing
pkg-installation-type: "frozen"

- name: Run unit tests
run: pytest -v

- name: Verify that we can build the package
run: python3 -m build
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,8 @@ conda activate deeprank2
pip install deeprank2
```

We also provide a frozen environment YML file located at `env/deeprank2_frozen.yml` with all dependencies set to fixed versions. The `env/deeprank2_frozen.yml` file provides a frozen environment with all dependencies set to fixed versions. This ensures reproducibility of experiments and results by preventing changes in package versions that could occur due to updates or modifications in the default `env/deeprank2.yml`. Use this frozen environment file for a stable and consistent setup, particularly if you encounter issues with the default environment file.

#### Manual installation (customizable)

If you want to use the GPUs, choose a specific python version (note that at the moment we support python 3.10 only), are a MacOS user, or if the YML installation was not successful, you can install the package manually. We advise to do this inside a [conda virtual environment](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html).
Expand Down
2 changes: 2 additions & 0 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ conda activate deeprank2
pip install deeprank2
```

We also provide a frozen environment YML file located at `env/deeprank2_frozen.yml` with all dependencies set to fixed versions. The `env/deeprank2_frozen.yml` file provides a frozen environment with all dependencies set to fixed versions. This ensures reproducibility of experiments and results by preventing changes in package versions that could occur due to updates or modifications in the default `env/deeprank2.yml`. Use this frozen environment file for a stable and consistent setup, particularly if you encounter issues with the default environment file.

#### Manual installation (customizable)

If you want to use the GPUs, choose a specific python version (note that at the moment we support python 3.10 only), are a MacOS user, or if the YML installation was not successful, you can install the package manually. We advise to do this inside a [conda virtual environment](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html).
Expand Down
10 changes: 6 additions & 4 deletions env/deeprank2.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,16 @@ channels:
- conda-forge
- sbl
dependencies:
- sbl::libcifpp>=5.1.0
- sbl::dssp>=4.2.2.1
- msms>=2.6.1
- python=3.10
- pip>=23.3
- sbl::libcifpp=5.1.0
- sbl::dssp=4.2.2.1
- msms=2.6.1
- pytorch=2.1.1
- torchvision>=0.16.1
- torchaudio>=2.1.1
- cpuonly>=2.0
- pyg>=2.4.0
- pyg=2.4.0
- pytorch-scatter>=2.1.2
- pytorch-sparse>=0.6.18
- pytorch-cluster>=1.6.3
Expand Down
234 changes: 234 additions & 0 deletions env/deeprank2_frozen.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
name: deeprank2
channels:
- pyg
- bioconda
- sbl
- pytorch
- defaults
- conda-forge
dependencies:
- _libgcc_mutex=0.1=conda_forge
- _openmp_mutex=4.5=2_gnu
- blas=1.0=mkl
- brotli-python=1.0.9=py310h6a678d5_8
- bzip2=1.0.8=h5eee18b_6
- ca-certificates=2024.3.11=h06a4308_0
- certifi=2024.6.2=py310h06a4308_0
- charset-normalizer=2.0.4=pyhd3eb1b0_0
- cpuonly=2.0=0
- dssp=4.2.2.1=h3fd9d12_0
- ffmpeg=4.3=hf484d3e_0
- filelock=3.13.1=py310h06a4308_0
- freetype=2.12.1=h4a9f257_0
- gmp=6.2.1=h295c915_3
- gmpy2=2.1.2=py310heeb90bb_0
- gnutls=3.6.15=he1e5248_0
- idna=3.7=py310h06a4308_0
- intel-openmp=2023.1.0=hdb19cb5_46306
- jinja2=3.1.4=py310h06a4308_0
- joblib=1.4.2=py310h06a4308_0
- jpeg=9e=h5eee18b_1
- lame=3.100=h7b6447c_0
- lcms2=2.12=h3be6417_0
- ld_impl_linux-64=2.38=h1181459_1
- lerc=3.0=h295c915_0
- libcifpp=5.1.0=hf65b397_0
- libdeflate=1.17=h5eee18b_1
- libffi=3.4.4=h6a678d5_1
- libgcc-ng=13.2.0=h77fa898_11
- libgfortran-ng=11.2.0=h00389a5_1
- libgfortran5=11.2.0=h1234567_1
- libgomp=13.2.0=h77fa898_11
- libiconv=1.16=h5eee18b_3
- libidn2=2.3.4=h5eee18b_0
- libjpeg-turbo=2.0.0=h9bf148f_0
- libmcfp=1.2.3=h3fd9d12_0
- libpng=1.6.39=h5eee18b_0
- libstdcxx-ng=13.2.0=hc0a3c3a_11
- libtasn1=4.19.0=h5eee18b_0
- libtiff=4.5.1=h6a678d5_0
- libunistring=0.9.10=h27cfd23_0
- libuuid=1.41.5=h5eee18b_0
- libwebp-base=1.3.2=h5eee18b_0
- libzlib=1.2.13=h4ab18f5_6
- llvm-openmp=14.0.6=h9e868ea_0
- lz4-c=1.9.4=h6a678d5_1
- markupsafe=2.1.3=py310h5eee18b_0
- mkl=2023.1.0=h213fc3f_46344
- mkl-service=2.4.0=py310h5eee18b_1
- mkl_fft=1.3.8=py310h5eee18b_0
- mkl_random=1.2.4=py310hdb19cb5_0
- mpc=1.1.0=h10f8cd9_1
- mpfr=4.0.2=hb69a4c5_1
- mpmath=1.3.0=py310h06a4308_0
- msms=2.6.1=h9ee0642_3
- ncurses=6.4=h6a678d5_0
- nettle=3.7.3=hbbd107a_1
- networkx=3.2.1=py310h06a4308_0
- numpy=1.26.4=py310h5f9d8c6_0
- numpy-base=1.26.4=py310hb5e798b_0
- openh264=2.1.1=h4ff587b_0
- openjpeg=2.4.0=h3ad879b_0
- openssl=3.0.14=h5eee18b_0
- pillow=10.3.0=py310h5eee18b_0
- pip=24.0=py310h06a4308_0
- psutil=5.9.0=py310h5eee18b_0
- pybind11-abi=4=hd3eb1b0_1
- pyg=2.4.0=py310_torch_2.1.0_cpu
- pyparsing=3.0.9=py310h06a4308_0
- pysocks=1.7.1=py310h06a4308_0
- python=3.10.14=h955ad1f_1
- pytorch=2.1.1=py3.10_cpu_0
- pytorch-cluster=1.6.3=py310_torch_2.1.0_cpu
- pytorch-mutex=1.0=cpu
- pytorch-scatter=2.1.2=py310_torch_2.1.0_cpu
- pytorch-sparse=0.6.18=py310_torch_2.1.0_cpu
- pytorch-spline-conv=1.2.2=py310_torch_2.1.0_cpu
- pyyaml=6.0.1=py310h5eee18b_0
- readline=8.2=h5eee18b_0
- requests=2.32.2=py310h06a4308_0
- scipy=1.13.1=py310h5f9d8c6_0
- setuptools=69.5.1=py310h06a4308_0
- sqlite=3.45.3=h5eee18b_0
- sympy=1.12=py310h06a4308_0
- tbb=2021.8.0=hdb19cb5_0
- tk=8.6.14=h39e8969_0
- torchaudio=2.1.1=py310_cpu
- torchvision=0.16.1=py310_cpu
- tqdm=4.66.4=py310h2f386ee_0
- typing_extensions=4.11.0=py310h06a4308_0
- urllib3=2.2.2=py310h06a4308_0
- wheel=0.43.0=py310h06a4308_0
- xz=5.4.6=h5eee18b_1
- yaml=0.2.5=h7b6447c_0
- zlib=1.2.13=h4ab18f5_6
- zstd=1.5.5=hc292b87_2
- pip:
- absl-py==2.1.0
- anyio==4.4.0
- argon2-cffi==23.1.0
- argon2-cffi-bindings==21.2.0
- arrow==1.3.0
- asttokens==2.4.1
- async-lru==2.0.4
- attrs==23.2.0
- babel==2.15.0
- beautifulsoup4==4.12.3
- biopython==1.83
- bleach==6.1.0
- blosc2==2.7.0
- bump2version==1.0.1
- cffi==1.16.0
- chart-studio==1.1.0
- comm==0.2.2
- contourpy==1.2.1
- coverage==6.5.0
- coveralls==3.3.1
- cycler==0.12.1
- debugpy==1.8.1
- decorator==5.1.1
- defusedxml==0.7.1
- dill==0.3.8
- docopt==0.6.2
- exceptiongroup==1.2.1
- executing==2.0.1
- fastjsonschema==2.20.0
- fonttools==4.53.0
- fqdn==1.5.1
- freesasa==2.2.1
- grpcio==1.64.1
- h11==0.14.0
- h5py==3.11.0
- httpcore==1.0.5
- httpx==0.27.0
- iniconfig==2.0.0
- ipykernel==6.29.4
- ipython==8.25.0
- isoduration==20.11.0
- jedi==0.19.1
- json5==0.9.25
- jsonpointer==3.0.0
- jsonschema==4.22.0
- jsonschema-specifications==2023.12.1
- jupyter-client==8.6.2
- jupyter-core==5.7.2
- jupyter-events==0.10.0
- jupyter-lsp==2.2.5
- jupyter-server==2.14.1
- jupyter-server-terminals==0.5.3
- jupyterlab==4.2.2
- jupyterlab-pygments==0.3.0
- jupyterlab-server==2.27.2
- kiwisolver==1.4.5
- markdown==3.6
- markov-clustering==0.0.6.dev0
- matplotlib==3.9.0
- matplotlib-inline==0.1.7
- mistune==3.0.2
- msgpack==1.0.8
- nbclient==0.10.0
- nbconvert==7.16.4
- nbformat==5.10.4
- ndindex==1.8
- nest-asyncio==1.6.0
- notebook==7.2.1
- notebook-shim==0.2.4
- numexpr==2.10.1
- overrides==7.7.0
- packaging==24.1
- pandas==2.2.2
- pandocfilters==1.5.1
- parso==0.8.4
- pdb2sql==0.5.3
- pexpect==4.9.0
- platformdirs==4.2.2
- plotly==5.22.0
- pluggy==1.5.0
- prometheus-client==0.20.0
- prompt-toolkit==3.0.47
- protobuf==4.25.3
- ptyprocess==0.7.0
- pure-eval==0.2.2
- py-cpuinfo==9.0.0
- pyarrow==16.1.0
- pycodestyle==2.12.0
- pycparser==2.22
- pygments==2.18.0
- pytest==7.4.4
- pytest-cov==4.1.0
- pytest-runner==6.0.1
- python-dateutil==2.9.0.post0
- python-json-logger==2.0.7
- python-louvain==0.16
- pytz==2024.1
- pyzmq==26.0.3
- referencing==0.35.1
- retrying==1.3.4
- rfc3339-validator==0.1.4
- rfc3986-validator==0.1.1
- rpds-py==0.18.1
- ruff==0.4.10
- scikit-learn==1.5.0
- send2trash==1.8.3
- six==1.16.0
- sniffio==1.3.1
- soupsieve==2.5
- stack-data==0.6.3
- tables==3.9.2
- tenacity==8.4.1
- tensorboard==2.17.0
- tensorboard-data-server==0.7.2
- terminado==0.18.1
- threadpoolctl==3.5.0
- tinycss2==1.3.0
- tomli==2.0.1
- tornado==6.4.1
- traitlets==5.14.3
- types-python-dateutil==2.9.0.20240316
- tzdata==2024.1
- uri-template==1.3.0
- wcwidth==0.2.13
- webcolors==24.6.0
- webencodings==0.5.1
- websocket-client==1.8.0
- werkzeug==3.0.3
Loading
Loading