Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add XPU to binary build generation #5489

Merged
merged 4 commits into from
Jul 23, 2024
Merged

Conversation

chuanqi129
Copy link
Contributor

@chuanqi129 chuanqi129 commented Jul 22, 2024

Torch XPU nightly wheel build has been enabled with #pytorch/pytorch#129560 landed, add XPU to binary build generation. Works for pytorch/pytorch#114850

> python tools/scripts/generate_binary_build_matrix.py --with-xpu enable
{
  "include": [
    {
      "python_version": "3.8",
      "gpu_arch_type": "cpu",
      "gpu_arch_version": "",
      "desired_cuda": "cpu",
      "container_image": "pytorch/manylinux-builder:cpu",
      "package_type": "manywheel",
      "build_name": "manywheel-py3_8-cpu",
      "validation_runner": "linux.2xlarge",
      "installation": "pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cpu",
      "channel": "nightly",
      "upload_to_base_bucket": "no",
      "stable_version": "2.3.1",
      "use_split_build": false
    },
    {
      "python_version": "3.8",
      "gpu_arch_type": "cuda",
      "gpu_arch_version": "11.8",
      "desired_cuda": "cu118",
      "container_image": "pytorch/manylinux-builder:cuda11.8",
      "package_type": "manywheel",
      "build_name": "manywheel-py3_8-cuda11_8",
      "validation_runner": "linux.g5.4xlarge.nvidia.gpu",
      "installation": "pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu118",
      "channel": "nightly",
      "upload_to_base_bucket": "no",
      "stable_version": "2.3.1",
      "use_split_build": false
    },
    {
      "python_version": "3.8",
      "gpu_arch_type": "cuda",
      "gpu_arch_version": "12.1",
      "desired_cuda": "cu121",
      "container_image": "pytorch/manylinux-builder:cuda12.1",
      "package_type": "manywheel",
      "build_name": "manywheel-py3_8-cuda12_1",
      "validation_runner": "linux.g5.4xlarge.nvidia.gpu",
      "installation": "pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121",
      "channel": "nightly",
      "upload_to_base_bucket": "no",
      "stable_version": "2.3.1",
      "use_split_build": false
    },
    {
      "python_version": "3.8",
      "gpu_arch_type": "cuda",
      "gpu_arch_version": "12.4",
      "desired_cuda": "cu124",
      "container_image": "pytorch/manylinux-builder:cuda12.4",
      "package_type": "manywheel",
      "build_name": "manywheel-py3_8-cuda12_4",
      "validation_runner": "linux.g5.4xlarge.nvidia.gpu",
      "installation": "pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu124",
      "channel": "nightly",
      "upload_to_base_bucket": "no",
      "stable_version": "2.3.1",
      "use_split_build": false
    },
    {
      "python_version": "3.8",
      "gpu_arch_type": "rocm",
      "gpu_arch_version": "6.0",
      "desired_cuda": "rocm6.0",
      "container_image": "pytorch/manylinux-builder:rocm6.0",
      "package_type": "manywheel",
      "build_name": "manywheel-py3_8-rocm6_0",
      "validation_runner": "linux.2xlarge",
      "installation": "pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.0",
      "channel": "nightly",
      "upload_to_base_bucket": "no",
      "stable_version": "2.3.1",
      "use_split_build": false
    },
    {
      "python_version": "3.8",
      "gpu_arch_type": "rocm",
      "gpu_arch_version": "6.1",
      "desired_cuda": "rocm6.1",
      "container_image": "pytorch/manylinux-builder:rocm6.1",
      "package_type": "manywheel",
      "build_name": "manywheel-py3_8-rocm6_1",
      "validation_runner": "linux.2xlarge",
      "installation": "pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.1",
      "channel": "nightly",
      "upload_to_base_bucket": "no",
      "stable_version": "2.3.1",
      "use_split_build": false
    },
    {
      "python_version": "3.8",
      "gpu_arch_type": "xpu",
      "gpu_arch_version": "",
      "desired_cuda": "xpu",
      "container_image": "pytorch/manylinux2_28-builder:xpu",
      "package_type": "manywheel",
      "build_name": "manywheel-py3_8-xpu",
      "validation_runner": "linux.2xlarge",
      "installation": "pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/xpu",
      "channel": "nightly",
      "upload_to_base_bucket": "no",
      "stable_version": "2.3.1",
      "use_split_build": false
    }
  ]
}

Copy link

vercel bot commented Jul 22, 2024

@chuanqi129 is attempting to deploy a commit to the Meta Open Source Team on Vercel.

A member of the Team first needs to authorize it.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 22, 2024
@@ -35,6 +35,10 @@ on:
description: "Build with CPU?"
default: "enable"
type: string
with-xpu:
description: "Build with XPU?"
default: "enable"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please keep it disable for now. Since this will automatically enable vision and audio builds. I don't think we want this at this point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @atalman thanks for the review, we plan to enable vision and audio build also for now. So that we can have completed nightly install cmd like pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/xpu. How do you think? Any concern in here?

Copy link
Contributor

@atalman atalman Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, thanks @atalman for the reminder. Got your point now, I will set the with_xpu disable as default and enable it in torchvision and torchaudio after this PR land, does it make sense to you?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed

@chuanqi129 chuanqi129 requested a review from atalman July 22, 2024 23:50
Copy link
Contributor

@atalman atalman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, please adress comment of setting with with-xpu to disable by default

@atalman atalman merged commit 7880593 into pytorch:main Jul 23, 2024
116 of 123 checks passed
atalman added a commit that referenced this pull request Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants