Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling of PyTorch dependency #210

Open
jakirkham opened this issue Aug 23, 2024 · 0 comments
Open

Handling of PyTorch dependency #210

jakirkham opened this issue Aug 23, 2024 · 0 comments

Comments

@jakirkham
Copy link
Member

jakirkham commented Aug 23, 2024

During a recent review of dependencies here we noticed that PyTorch is a dependency, but is not listed as one at runtime

run:
- {{ pin_compatible('cuda-version', max_pin='x', min_pin='x') }}
{% if cuda_major == "11" %}
- cudatoolkit
{% endif %}
- libwholegraph ={{ version }}
- python

However there are a range of tightly pinned PyTorch builds (on the older side). All of them for CUDA 11. No CUDA 12

specific:
- output_types: [conda]
matrices:
- matrix:
arch: x86_64
cuda: "11.2"
packages:
# It's impossible to create this environment with pyg because
# the pyg package has an explicit dependency on cudatoolkit=11.*
# and there simply isn't any build for cudatoolkit=11.2.
# Note that the packages for CUDA 11.2/11.4 environments are the
# ones from conda-forge (built only against CUDA 11.2) and
# *not* the pytorch channel. For CUDA 11.5/11.8 environments,
# we're using packages from the pytorch channel.
- pytorch=1.11.0=*cuda112*
- matrix:
arch: x86_64
cuda: "11.4"
packages:
# It's impossible to create this environment with pyg because
# the pyg package has an explicit dependency on cudatoolkit=11.*
# and there simply isn't any build for cudatoolkit=11.4.
# There is also no build of pytorch for CUDA 11.4 but the 11.2
# build should work in practice and doesn't require any
# cudatoolkit version explicitly.
- pytorch=1.11.0=*cuda112*
- matrix:
arch: x86_64
cuda: "11.5"
packages:
# This environment "just works" for both pytorch and pyg, but only
# with older pytorch versions since the newest ones aren't built
# against 11.5 anymore.
- pytorch=1.11.0=*cuda11.5*
- matrix:
arch: x86_64
cuda: "11.8"
packages:
# Since CUDA 11.6, pytorch switched to using the `cuda-*` packages
# as dependencies for its official conda package. These are only
# available from the nvidia channel at the moment, and this will
# probably continue once conda-forge has added these new packages
# since conda-forge will only add this from CUDA 12.0 onwards,
# at least in the near-term.
# Our own RAPIDS packages are dependent on the `cudatoolkit`
# package from conda-forge though, which means that we have to
# install both `cudatoolkit` version 11.8 and the `cuda-*` packages
# version 11.8 here.
# Starting with Pytorch 2.0, this works well though, since Pytorch
# has largely reduced its dependencies, so only part of the CUDA
# toolkit needs to be duplicated this way.
# If conda-forge supports the new cuda-* packages for CUDA 11.8
# at some point, then we can fully support/properly specify
# this environment.
- pytorch=2.0.0
- pytorch-cuda=11.8
- matrix:
arch: aarch64
cuda: "11.8"
packages:
- pytorch=2.0.0
- pytorch-cuda=11.8

Not really sure how to handle updating PyTorch here or how it should be reflected in dependencies. So opening this issue to discuss and find a reasonable resolution

@jakirkham jakirkham changed the title Handling of PyTorch Handling of PyTorch dependency Aug 23, 2024
This was referenced Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant