Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove NumPy <2 pin #4615

Merged
merged 27 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
acafdae
Remove NumPy <2 pin.
seberg Aug 19, 2024
df1f21d
Include pre-release identifier for upper pin
seberg Aug 20, 2024
7d6267e
Merge branch 'branch-24.10' into my_new_branch
seberg Aug 23, 2024
287d629
Merge branch 'branch-24.10' into my_new_branch
jakirkham Aug 24, 2024
d54b7d8
Try removing torch pin from tests, add torchdata/pydantic explicitly
seberg Aug 24, 2024
3384bc6
Readd PyTorch and relax upperbound
jakirkham Aug 27, 2024
37d2ca0
Merge branch 'branch-24.10' into my_new_branch
jakirkham Aug 27, 2024
21b689b
Updates test to ensure a query string is created using integer vertex…
rlratzel Aug 28, 2024
d898b84
Cleanup based on review feedback: uses a list comprehension instead o…
rlratzel Aug 28, 2024
1d66490
Merge branch 'branch-24.10' into my_new_branch
alexbarghi-nv Aug 28, 2024
ba6cbab
Merge branch 'branch-24.10' into my_new_branch
jakirkham Aug 29, 2024
6fa8d19
Merge branch 'branch-24.10' into my_new_branch
seberg Sep 5, 2024
0e37bda
Merge branch 'branch-24.10' into my_new_branch
jameslamb Sep 17, 2024
a7d9a6b
Merge remote-tracking branch 'upstream/branch-24.10' into my_new_branch
hcho3 Sep 19, 2024
f74daa3
Force install Numpy 2.0
hcho3 Sep 20, 2024
3fea26d
Merge remote-tracking branch 'upstream/branch-24.10' into numpy20_sup…
hcho3 Sep 20, 2024
5e88901
Pin pytorch<2.4
hcho3 Sep 21, 2024
2929e81
Add pin pytorch<2.4
hcho3 Sep 23, 2024
16b457d
Reviewer's comment
hcho3 Sep 24, 2024
cecafb4
Update Conda recipes
hcho3 Sep 24, 2024
efdff0a
Merge remote-tracking branch 'upstream/branch-24.10' into numpy20_sup…
hcho3 Sep 24, 2024
7926695
Merge branch 'branch-24.10' into my_new_branch
jameslamb Sep 25, 2024
a49f912
Use pytorch 2.3.0
hcho3 Sep 25, 2024
195c5df
Merge remote-tracking branch 'upstream/branch-24.10' into numpy20_sup…
hcho3 Sep 26, 2024
7d7d042
Bump nccl; update torch pins
hcho3 Sep 26, 2024
d818cf0
Fix
hcho3 Sep 26, 2024
1320573
Merge branch 'branch-24.10' into my_new_branch
jakirkham Sep 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions ci/test_python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,8 @@ if [[ "${RAPIDS_CUDA_VERSION}" == "11.8.0" ]]; then
cugraph \
cugraph-dgl \
'dgl>=1.1.0.cu*,<=2.0.0.cu*' \
'pytorch>=2.0' \
'pytorch>=2.0,<2.4' \
'numpy>=2.0' \
hcho3 marked this conversation as resolved.
Show resolved Hide resolved
hcho3 marked this conversation as resolved.
Show resolved Hide resolved
'cuda-version=11.8'

rapids-print-env
Expand Down Expand Up @@ -210,7 +211,8 @@ if [[ "${RAPIDS_CUDA_VERSION}" == "11.8.0" ]]; then
--channel "${PYTHON_CHANNEL}" \
--channel pyg \
"cugraph-pyg" \
"ogb"
"ogb" \
"numpy>=2.0"

pip install \
pyg_lib \
Expand Down Expand Up @@ -253,7 +255,8 @@ if [[ "${RAPIDS_CUDA_VERSION}" == "11.8.0" ]]; then
--channel "${PYTHON_CHANNEL}" \
--channel conda-forge \
--channel nvidia \
cugraph-equivariant
cugraph-equivariant \
"numpy>=2.0"
pip install e3nn==0.5.1

rapids-print-env
Expand Down
12 changes: 1 addition & 11 deletions ci/test_wheel_cugraph-dgl.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,18 +32,8 @@ fi
PYTORCH_URL="https://download.pytorch.org/whl/cu${PYTORCH_CUDA_VER}"
DGL_URL="https://data.dgl.ai/wheels/cu${PYTORCH_CUDA_VER}/repo.html"

# Starting from 2.2, PyTorch wheels depend on nvidia-nccl-cuxx>=2.19 wheel and
# dynamically link to NCCL. RAPIDS CUDA 11 CI images have an older NCCL version that
# might shadow the newer NCCL required by PyTorch during import (when importing
# `cupy` before `torch`).
if [[ "${NCCL_VERSION}" < "2.19" ]]; then
PYTORCH_VER="2.1.0"
else
PYTORCH_VER="2.3.0"
fi
Comment on lines -35 to -43
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We recently bumped the NCCL floor to 2.18.1.1 in PR: #4661

Reading this comment, am wondering whether we need to bump it further (namely to 2.19)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Raising in the NCCL update issue: rapidsai/build-planning#102 (comment)


rapids-logger "Installing PyTorch and DGL"
rapids-retry python -m pip install "torch==${PYTORCH_VER}" --index-url ${PYTORCH_URL}
rapids-retry python -m pip install torch==2.3.0 --index-url ${PYTORCH_URL}
rapids-retry python -m pip install dgl==2.0.0 --find-links ${DGL_URL}

python -m pytest python/cugraph-dgl/tests
6 changes: 3 additions & 3 deletions ci/test_wheel_cugraph-pyg.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,13 @@ export CI_RUN=1

if [[ "${CUDA_VERSION}" == "11.8.0" ]]; then
PYTORCH_URL="https://download.pytorch.org/whl/cu118"
PYG_URL="https://data.pyg.org/whl/torch-2.1.0+cu118.html"
PYG_URL="https://data.pyg.org/whl/torch-2.3.0+cu118.html"
else
PYTORCH_URL="https://download.pytorch.org/whl/cu121"
PYG_URL="https://data.pyg.org/whl/torch-2.1.0+cu121.html"
PYG_URL="https://data.pyg.org/whl/torch-2.3.0+cu121.html"
fi
rapids-logger "Installing PyTorch and PyG dependencies"
rapids-retry python -m pip install torch==2.1.0 --index-url ${PYTORCH_URL}
rapids-retry python -m pip install torch==2.3.0 --index-url ${PYTORCH_URL}
rapids-retry python -m pip install "torch-geometric>=2.5,<2.6"
rapids-retry python -m pip install \
ogb \
Expand Down
4 changes: 2 additions & 2 deletions conda/environments/all_cuda-118_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ dependencies:
- ninja
- notebook>=0.5.0
- numba>=0.57
- numpy>=1.23,<2.0a0
- numpy>=1.23,<3.0a0
- numpydoc
- nvcc_linux-64=11.8
- ogb
Expand All @@ -57,7 +57,7 @@ dependencies:
- pytest-mpl
- pytest-xdist
- python-louvain
- pytorch>=2.0,<2.2.0a0
- pytorch>=2.0,<2.4.0a0
- raft-dask==24.10.*,>=0.0.0a0
- rapids-build-backend>=0.3.1,<0.4.0.dev0
- rapids-dask-dependency==24.10.*,>=0.0.0a0
Expand Down
4 changes: 2 additions & 2 deletions conda/environments/all_cuda-125_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ dependencies:
- ninja
- notebook>=0.5.0
- numba>=0.57
- numpy>=1.23,<2.0a0
- numpy>=1.23,<3.0a0
- numpydoc
- ogb
- openmpi
Expand All @@ -62,7 +62,7 @@ dependencies:
- pytest-mpl
- pytest-xdist
- python-louvain
- pytorch>=2.0,<2.2.0a0
- pytorch>=2.0,<2.4.0a0
- raft-dask==24.10.*,>=0.0.0a0
- rapids-build-backend>=0.3.1,<0.4.0.dev0
- rapids-dask-dependency==24.10.*,>=0.0.0a0
Expand Down
4 changes: 2 additions & 2 deletions conda/recipes/cugraph-dgl/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ requirements:
- cugraph ={{ version }}
- dgl >=1.1.0.cu*
- numba >=0.57
- numpy >=1.23,<2.0a0
- numpy >=1.23,<3.0a0
- pylibcugraphops ={{ minor_version }}
- tensordict >=0.1.2
- python
- pytorch >=2.0
- pytorch >=2.0,<2.4.0a0
- cupy >=12.0.0

tests:
Expand Down
4 changes: 2 additions & 2 deletions conda/recipes/cugraph-pyg/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ requirements:
run:
- rapids-dask-dependency ={{ minor_version }}
- numba >=0.57
- numpy >=1.23,<2.0a0
- numpy >=1.23,<3.0a0
- python
- pytorch >=2.0
- pytorch >=2.0,<2.4.0a0
- cupy >=12.0.0
- cugraph ={{ version }}
- pylibcugraphops ={{ minor_version }}
Expand Down
2 changes: 1 addition & 1 deletion conda/recipes/cugraph-service/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ outputs:
- dask-cuda ={{ minor_version }}
- dask-cudf ={{ minor_version }}
- numba >=0.57
- numpy >=1.23,<2.0a0
- numpy >=1.23,<3.0a0
- python
- rapids-dask-dependency ={{ minor_version }}
- thriftpy2 >=0.4.15,!=0.5.0,!=0.5.1
Expand Down
12 changes: 7 additions & 5 deletions dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -524,7 +524,7 @@ dependencies:
- &dask rapids-dask-dependency==24.10.*,>=0.0.0a0
- &dask_cuda dask-cuda==24.10.*,>=0.0.0a0
- &numba numba>=0.57
- &numpy numpy>=1.23,<2.0a0
- &numpy numpy>=1.23,<3.0a0
- output_types: conda
packages:
- aiohttp
Expand Down Expand Up @@ -695,7 +695,9 @@ dependencies:
- output_types: [conda]
packages:
- *cugraph_unsuffixed
- pytorch>=2.0
# ceiling could be removed when this is fixed:
# https://github.com/conda-forge/pytorch-cpu-feedstock/issues/254
- &pytorch_conda pytorch>=2.0,<2.4.0a0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to bump the floor on these as well?

- pytorch-cuda==11.8
- &tensordict tensordict>=0.1.2
- dgl>=1.1.0.cu*
Expand All @@ -704,7 +706,7 @@ dependencies:
- output_types: [conda]
packages:
- *cugraph_unsuffixed
- pytorch>=2.0
- *pytorch_conda
- pytorch-cuda==11.8
- *tensordict
- pyg>=2.5,<2.6
Expand All @@ -713,7 +715,7 @@ dependencies:
common:
- output_types: [conda]
packages:
- &pytorch_unsuffixed pytorch>=2.0,<2.2.0a0
- *pytorch_conda
- torchdata
- pydantic
- ogb
Expand All @@ -733,7 +735,7 @@ dependencies:
matrices:
- matrix: {cuda: "12.*"}
packages:
- &pytorch_pip torch>=2.0,<2.2.0a0
- &pytorch_pip torch>=2.0,<2.4.0a0
- *tensordict
- matrix: {cuda: "11.*"}
packages:
Expand Down
2 changes: 1 addition & 1 deletion python/cugraph-dgl/conda/cugraph_dgl_dev_cuda-118.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ dependencies:
- pytest-cov
- pytest-xdist
- pytorch-cuda==11.8
- pytorch>=2.0
- pytorch>=2.0,<2.4.0a0
- scipy
- tensordict>=0.1.2
name: cugraph_dgl_dev_cuda-118
4 changes: 2 additions & 2 deletions python/cugraph-dgl/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ classifiers = [
dependencies = [
"cugraph==24.10.*,>=0.0.0a0",
"numba>=0.57",
"numpy>=1.23,<2.0a0",
"numpy>=1.23,<3.0a0",
"pylibcugraphops==24.10.*,>=0.0.0a0",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.

Expand All @@ -40,7 +40,7 @@ test = [
"pytest-xdist",
"scipy",
"tensordict>=0.1.2",
"torch>=2.0,<2.2.0a0",
"torch>=2.0,<2.4.0a0",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.

[project.urls]
Expand Down
2 changes: 1 addition & 1 deletion python/cugraph-pyg/conda/cugraph_pyg_dev_cuda-118.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ dependencies:
- pytest-cov
- pytest-xdist
- pytorch-cuda==11.8
- pytorch>=2.0
- pytorch>=2.0,<2.4.0a0
- scipy
- tensordict>=0.1.2
name: cugraph_pyg_dev_cuda-118
4 changes: 2 additions & 2 deletions python/cugraph-pyg/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ classifiers = [
dependencies = [
"cugraph==24.10.*,>=0.0.0a0",
"numba>=0.57",
"numpy>=1.23,<2.0a0",
"numpy>=1.23,<3.0a0",
"pylibcugraphops==24.10.*,>=0.0.0a0",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.

Expand All @@ -49,7 +49,7 @@ test = [
"pytest-xdist",
"scipy",
"tensordict>=0.1.2",
"torch>=2.0,<2.2.0a0",
"torch>=2.0,<2.4.0a0",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.

[tool.setuptools]
Expand Down
4 changes: 2 additions & 2 deletions python/cugraph-service/server/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ dependencies = [
"dask-cuda==24.10.*,>=0.0.0a0",
"dask-cudf==24.10.*,>=0.0.0a0",
"numba>=0.57",
"numpy>=1.23,<2.0a0",
"numpy>=1.23,<3.0a0",
"rapids-dask-dependency==24.10.*,>=0.0.0a0",
"rmm==24.10.*,>=0.0.0a0",
"thriftpy2!=0.5.0,!=0.5.1",
Expand All @@ -47,7 +47,7 @@ cugraph-service-server = "cugraph_service_server.__main__:main"
[project.optional-dependencies]
test = [
"networkx>=2.5.1",
"numpy>=1.23,<2.0a0",
"numpy>=1.23,<3.0a0",
"pandas",
"pytest",
"pytest-benchmark",
Expand Down
14 changes: 8 additions & 6 deletions python/cugraph/cugraph/tests/data_store/test_property_graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -2576,9 +2576,10 @@ def bench_extract_subgraph_for_rmat(gpubenchmark, rmat_PropertyGraph):
scn = PropertyGraph.src_col_name
dcn = PropertyGraph.dst_col_name

verts = []
for i in range(0, 10000, 10):
verts.append(generated_df["src"].iloc[i])
# Build a query string to extract a graph with only specific edges based on
# the integer vertex IDs. Other edge and/or vertex properties can be
# included in the query as well.
verts = [int(generated_df["src"].iloc[i]) for i in range(0, 10000, 10)]

selected_edges = pG.select_edges(f"{scn}.isin({verts}) | {dcn}.isin({verts})")
gpubenchmark(
Expand Down Expand Up @@ -2618,9 +2619,10 @@ def bench_extract_subgraph_for_rmat_detect_duplicate_edges(
scn = PropertyGraph.src_col_name
dcn = PropertyGraph.dst_col_name

verts = []
for i in range(0, 10000, 10):
verts.append(generated_df["src"].iloc[i])
# Build a query string to extract a graph with only specific edges based on
# the integer vertex IDs. Other edge and/or vertex properties can be
# included in the query as well.
verts = [int(generated_df["src"].iloc[i]) for i in range(0, 10000, 10)]

selected_edges = pG.select_edges(f"{scn}.isin({verts}) | {dcn}.isin({verts})")

Expand Down
4 changes: 2 additions & 2 deletions python/cugraph/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ dependencies = [
"dask-cudf==24.10.*,>=0.0.0a0",
"fsspec[http]>=0.6.0",
"numba>=0.57",
"numpy>=1.23,<2.0a0",
"numpy>=1.23,<3.0a0",
"pylibcugraph==24.10.*,>=0.0.0a0",
"raft-dask==24.10.*,>=0.0.0a0",
"rapids-dask-dependency==24.10.*,>=0.0.0a0",
Expand All @@ -47,7 +47,7 @@ classifiers = [
[project.optional-dependencies]
test = [
"networkx>=2.5.1",
"numpy>=1.23,<2.0a0",
"numpy>=1.23,<3.0a0",
"pandas",
"pylibwholegraph==24.10.*,>=0.0.0a0",
"pytest",
Expand Down
2 changes: 1 addition & 1 deletion python/nx-cugraph/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ classifiers = [
dependencies = [
"cupy-cuda11x>=12.0.0",
"networkx>=3.0",
"numpy>=1.23,<2.0a0",
"numpy>=1.23,<3.0a0",
"pylibcugraph==24.10.*,>=0.0.0a0",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.

Expand Down
2 changes: 1 addition & 1 deletion python/pylibcugraph/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ classifiers = [
[project.optional-dependencies]
test = [
"cudf==24.10.*,>=0.0.0a0",
"numpy>=1.23,<2.0a0",
"numpy>=1.23,<3.0a0",
"pandas",
"pytest",
"pytest-benchmark",
Expand Down
Loading