Skip to content

Commit

Permalink
merge amd-mainline into release/rocm-rel-6.3
Browse files Browse the repository at this point in the history 
[SWDEV-493184][HIPIFY] Add the missing shuffle device functions support

Dedicated to the 6.3.1 release

Change-Id: I88fd986736da27695e11408189aebae87edf260e
  • Loading branch information
@emankov
emankov committed Nov 15, 2024
2 parents f156b77 + 4c14b3a commit 1b2b520
Show file tree
Hide file tree
Showing 28 changed files with 8,770 additions and 2,611 deletions.
142 changes: 78 additions & 64 deletions CHANGELOG.md
View file

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -272,9 +272,9 @@ if(HIPIFY_CLANG_TESTS OR HIPIFY_CLANG_TESTS_ONLY)
set (HIPIFY_CLANG_RES "${LLVM_LIBRARY_DIRS}/clang/${LIB_CLANG_RES}")

if(${CMAKE_VERSION} VERSION_LESS "3.27.0")
find_package(PythonInterp 2.7 REQUIRED)
find_package(PythonInterp 3.0 REQUIRED)
else()
find_package(Python 2.7...3.13 REQUIRED)
find_package(Python 3.0...3.14 REQUIRED)
endif()

function (require_program PROGRAM_NAME)
Expand Down
4,737 changes: 3,422 additions & 1,315 deletions bin/hipify-perl
View file

Large diffs are not rendered by default.

122 changes: 62 additions & 60 deletions docs/hipify-clang.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@ Dependencies

* `LLVM+Clang <http://releases.llvm.org>`_ of at least version
`4.0.0 <http://releases.llvm.org/download.html#4.0.0>`_; the latest stable and recommended release:
`19.1.1 <https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.1>`_.
`19.1.3 <https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.3>`_.

* `CUDA <https://developer.nvidia.com/cuda-downloads>`_ of at least version
`7.0 <https://developer.nvidia.com/cuda-toolkit-70>`_, the latest supported version is
`12.6.1 <https://developer.nvidia.com/cuda-downloads>`_.
`12.6.2 <https://developer.nvidia.com/cuda-downloads>`_.

.. list-table::

Expand Down Expand Up @@ -185,8 +185,10 @@ Dependencies
- ✅
- ✅
* - `19.1.0 <https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.0>`_,
`19.1.1 <https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.1>`_:sup:`4`
- `12.6.1 <https://developer.nvidia.com/cuda-downloads>`_:sup:`4`
`19.1.1 <https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.1>`_,
`19.1.2 <https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.2>`_,
`19.1.3 <https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.3>`_:sup:`4`
- `12.6.2 <https://developer.nvidia.com/cuda-downloads>`_:sup:`4`
- **Latest stable config**
- **Latest stable config**

Expand Down Expand Up @@ -228,7 +230,7 @@ Dependencies
In most cases, you can get a suitable version of ``LLVM+Clang`` with your package manager. However, you can also
`download a release archive <http://releases.llvm.org/>`_ and build or install it. In case of multiple versions of ``LLVM`` installed, set
`CMAKE_PREFIX_PATH <https://cmake.org/cmake/help/latest/variable/CMAKE_PREFIX_PATH.html>`_ so that
``CMake`` can find the desired version of ``LLVM``. For example, ``-DCMAKE_PREFIX_PATH=D:\LLVM\19.1.1\dist``.
``CMake`` can find the desired version of ``LLVM``. For example, ``-DCMAKE_PREFIX_PATH=D:\LLVM\19.1.3\dist``.

Usage
============================================================
Expand Down Expand Up @@ -261,7 +263,7 @@ header files used during the hipification process:

.. code:: shell
./hipify-clang square.cu --cuda-path=/usr/local/cuda-12.6 --clang-resource-directory=/usr/llvm/19.1.1/dist/lib/clang/19
./hipify-clang square.cu --cuda-path=/usr/local/cuda-12.6 --clang-resource-directory=/usr/llvm/19.1.3/dist/lib/clang/19
For more information, refer to the `Clang manual for compiling CUDA <https://llvm.org/docs/CompileCudaWithLLVM.html#compiling-cuda-code>`_.

Expand Down Expand Up @@ -398,7 +400,7 @@ To ensure LLVM being found or in case of multiple LLVM instances, specify the pa

.. code-block:: bash
-DCMAKE_PREFIX_PATH=/usr/llvm/19.1.1/dist
-DCMAKE_PREFIX_PATH=/usr/llvm/19.1.3/dist
On Windows, specify the following option for CMake in the first place:
``-G "Visual Studio 17 2022"``.
Expand Down Expand Up @@ -472,7 +474,7 @@ LLVM <= 9.0.1
LLVM >= 10.0.0
-----------------

1. Download `LLVM project <https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.1>`_ sources.
1. Download `LLVM project <https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.3>`_ sources.

2. Build `LLVM project <http://llvm.org/docs/CMake.html>`_:

Expand Down Expand Up @@ -546,7 +548,7 @@ LLVM >= 10.0.0

.. code-block:: shell
-DCUDA_DNN_ROOT_DIR=D:/CUDA/cuDNN/9.5.0
-DCUDA_DNN_ROOT_DIR=D:/CUDA/cuDNN/9.5.1
5. [Optional] Install `CUB 1.9.8 <https://github.com/NVIDIA/cub/releases/tag/1.9.8>`_ for ``CUDA < 11.0`` only;
for ``CUDA >= 11.0``, the CUB shipped with CUDA will be used for testing.
Expand All @@ -565,7 +567,7 @@ LLVM >= 10.0.0
-DCUDA_CUB_ROOT_DIR=D:/CUDA/CUB
6. Install `Python <https://www.python.org/downloads>`_ version 2.7 or greater.
6. Install `Python <https://www.python.org/downloads>`_ version 3.0 or greater.

7. Install ``lit`` and ``FileCheck``; these are distributed with LLVM.

Expand All @@ -575,13 +577,13 @@ LLVM >= 10.0.0

.. code-block:: bash
python /usr/llvm/19.1.1/llvm-project/llvm/utils/lit/setup.py install
python /usr/llvm/19.1.3/llvm-project/llvm/utils/lit/setup.py install
**Windows**:

.. code-block:: shell
python D:/LLVM/19.1.1/llvm-project/llvm/utils/lit/setup.py install
python D:/LLVM/19.1.3/llvm-project/llvm/utils/lit/setup.py install
In case of errors similar to ``ModuleNotFoundError: No module named 'setuptools'``, upgrade the ``setuptools`` package:

Expand All @@ -595,23 +597,23 @@ LLVM >= 10.0.0

.. code-block:: bash
-DLLVM_EXTERNAL_LIT=/usr/llvm/19.1.1/build/bin/llvm-lit
-DLLVM_EXTERNAL_LIT=/usr/llvm/19.1.3/build/bin/llvm-lit
**Windows**:

.. code-block:: shell
-DLLVM_EXTERNAL_LIT=D:/LLVM/19.1.1/build/Release/bin/llvm-lit.py
-DLLVM_EXTERNAL_LIT=D:/LLVM/19.1.3/build/Release/bin/llvm-lit.py
* ``FileCheck``:

**Linux**:

Copy from ``/usr/llvm/19.1.1/build/bin/`` to ``CMAKE_INSTALL_PREFIX/dist/bin``.
Copy from ``/usr/llvm/19.1.3/build/bin/`` to ``CMAKE_INSTALL_PREFIX/dist/bin``.

**Windows**:

Copy from ``D:/LLVM/19.1.1/build/Release/bin`` to ``CMAKE_INSTALL_PREFIX/dist/bin``.
Copy from ``D:/LLVM/19.1.3/build/Release/bin`` to ``CMAKE_INSTALL_PREFIX/dist/bin``.

Alternatively, specify the path to ``FileCheck`` in the ``CMAKE_INSTALL_PREFIX`` option.

Expand All @@ -638,16 +640,16 @@ On Linux, the following configurations are tested:

* Ubuntu 14: LLVM 4.0.0 - 7.1.0, CUDA 7.0 - 9.0, cuDNN 5.0.5 - 7.6.5
* Ubuntu 16-19: LLVM 8.0.0 - 14.0.6, CUDA 7.0 - 10.2, cuDNN 5.1.10 - 8.0.5
* Ubuntu 20-21: LLVM 9.0.0 - 19.1.1, CUDA 7.0 - 12.6.1, cuDNN 5.1.10 - 9.5.0
* Ubuntu 22-23: LLVM 13.0.0 - 19.1.1, CUDA 7.0 - 12.6.1, cuDNN 8.0.5 - 9.5.0
* Ubuntu 20-21: LLVM 9.0.0 - 19.1.3, CUDA 7.0 - 12.6.2, cuDNN 5.1.10 - 9.5.1
* Ubuntu 22-23: LLVM 13.0.0 - 19.1.3, CUDA 7.0 - 12.6.2, cuDNN 8.0.5 - 9.5.1

Minimum build system requirements for the above configurations:

* CMake 3.16.8, GNU C/C++ 9.2, Python 3.0.

Recommended build system requirements:

* CMake 3.30.4, GNU C/C++ 13.2, Python 3.12.7.
* CMake 3.31.0, GNU C/C++ 13.2, Python 3.13.0.

Here's how to build ``hipify-clang`` with testing support on ``Ubuntu 23.10.01``:

Expand All @@ -657,10 +659,10 @@ Here's how to build ``hipify-clang`` with testing support on ``Ubuntu 23.10.01``
-DHIPIFY_CLANG_TESTS=ON \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=../dist \
-DCMAKE_PREFIX_PATH=/usr/llvm/19.1.1/dist \
-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-12.6.1 \
-DCUDA_DNN_ROOT_DIR=/usr/local/cudnn-9.5.0 \
-DLLVM_EXTERNAL_LIT=/usr/llvm/19.1.1/build/bin/llvm-lit \
-DCMAKE_PREFIX_PATH=/usr/llvm/19.1.3/dist \
-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-12.6.2 \
-DCUDA_DNN_ROOT_DIR=/usr/local/cudnn-9.5.1 \
-DLLVM_EXTERNAL_LIT=/usr/llvm/19.1.3/build/bin/llvm-lit \
../hipify
The corresponding successful output is:
Expand All @@ -684,30 +686,30 @@ The corresponding successful output is:
-- - Test hipify-clang : ON
-- - Is part of HIP SDK : OFF
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.13")
-- Found LLVM 19.1.1:
-- - CMake module path : /usr/llvm/19.1.1/dist/lib/cmake/llvm
-- - Clang include path : /usr/llvm/19.1.1/dist/include
-- - LLVM Include path : /usr/llvm/19.1.1/dist/include
-- - Binary path : /usr/llvm/19.1.1/dist/bin
-- Found LLVM 19.1.3:
-- - CMake module path : /usr/llvm/19.1.3/dist/lib/cmake/llvm
-- - Clang include path : /usr/llvm/19.1.3/dist/include
-- - LLVM Include path : /usr/llvm/19.1.3/dist/include
-- - Binary path : /usr/llvm/19.1.3/dist/bin
-- Linker detection: GNU ld
-- ---- The below configuring for hipify-clang testing only ----
-- Found Python: /usr/bin/python3.12 (found version "3.12.7") found components: Interpreter
-- Found Python: /usr/bin/python3.13 (found version "3.13.0") found components: Interpreter
-- Found lit: /usr/local/bin/lit
-- Found FileCheck: /GIT/LLVM/trunk/dist/FileCheck
-- Initial CUDA to configure:
-- - CUDA Toolkit path : /usr/local/cuda-12.6.1
-- - CUDA Toolkit path : /usr/local/cuda-12.6.2
-- - CUDA Samples path :
-- - cuDNN path : /usr/local/cudnn-9.5.0
-- - cuDNN path : /usr/local/cudnn-9.5.1
-- - CUB path :
-- Found CUDAToolkit: /usr/local/cuda-12.6.1/targets/x86_64-linux/include (found version "12.6.68")
-- Found CUDAToolkit: /usr/local/cuda-12.6.2/targets/x86_64-linux/include (found version "12.6.68")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found CUDA config:
-- - CUDA Toolkit path : /usr/local/cuda-12.6.1
-- - CUDA Toolkit path : /usr/local/cuda-12.6.2
-- - CUDA Samples path : OFF
-- - cuDNN path : /usr/local/cudnn-9.5.0
-- - CUB path : /usr/local/cuda-12.6.1/include/cub
-- - cuDNN path : /usr/local/cudnn-9.5.1
-- - CUB path : /usr/local/cuda-12.6.2/include/cub
-- Configuring done (0.5s)
-- Generating done (0.0s)
-- Build files have been written to: /usr/hipify/build
Expand All @@ -723,11 +725,11 @@ The corresponding successful output is:
Running HIPify regression tests
===============================================================
CUDA 12.6.68 - will be used for testing
LLVM 19.1.1 - will be used for testing
LLVM 19.1.3 - will be used for testing
x86_64 - Platform architecture
Linux 6.5.0-15-generic - Platform OS
64 - hipify-clang binary bitness
64 - python 3.12.7 binary bitness
64 - python 3.13.0 binary bitness
===============================================================
-- Testing: 106 tests, 12 threads --
Testing Time: 6.91s
Expand Down Expand Up @@ -819,16 +821,16 @@ Tested configurations:
- ``3.11.4``
* - ``17.0.1`` :sup:`6` - ``18.1.8`` :sup:`7`
- ``7.0 - 12.3.2``
- ``8.0.5 - 9.5.0``
- ``2019.16.11.40, 2022.17.11.4``
- ``3.30.4``
- ``3.12.7``
* - ``19.1.1``
- ``7.0 - 12.6.1``
- ``8.0.5 - 9.5.0``
- ``2019.16.11.40, 2022.17.11.4``
- ``3.30.4``
- ``3.12.7``
- ``8.0.5 - 9.5.1``
- ``2019.16.11.42, 2022.17.11.6``
- ``3.31.0``
- ``3.13.0``
* - ``19.1.0 - 19.1.3``
- ``7.0 - 12.6.2``
- ``8.0.5 - 9.5.1``
- ``2019.16.11.42, 2022.17.11.6``
- ``3.31.0``
- ``3.13.0``

:sup:`5` LLVM 14.x.x is the latest major release supporting Visual Studio 2017.

Expand All @@ -853,11 +855,11 @@ Building with testing support using ``Visual Studio 17 2022`` on ``Windows 11``:
-DHIPIFY_CLANG_TESTS=ON \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=../dist \
-DCMAKE_PREFIX_PATH=D:/LLVM/19.1.1/dist \
-DCMAKE_PREFIX_PATH=D:/LLVM/19.1.3/dist \
-DCUDA_TOOLKIT_ROOT_DIR="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6" \
-DCUDA_SDK_ROOT_DIR="C:/ProgramData/NVIDIA Corporation/CUDA Samples/v12.5" \
-DCUDA_DNN_ROOT_DIR=D:/CUDA/cuDNN/9.5.0 \
-DLLVM_EXTERNAL_LIT=D:/LLVM/19.1.1/build/Release/bin/llvm-lit.py \
-DCUDA_DNN_ROOT_DIR=D:/CUDA/cuDNN/9.5.1 \
-DLLVM_EXTERNAL_LIT=D:/LLVM/19.1.3/build/Release/bin/llvm-lit.py \
../hipify
The corresponding successful output is:
Expand All @@ -881,25 +883,25 @@ The corresponding successful output is:
-- - Build hipify-clang : ON
-- - Test hipify-clang : ON
-- - Is part of HIP SDK : OFF
-- Found LLVM 19.1.1:
-- - CMake module path : D:/LLVM/19.1.1/dist/lib/cmake/llvm
-- - Clang include path : D:/LLVM/19.1.1/dist/include
-- - LLVM Include path : D:/LLVM/19.1.1/dist/include
-- - Binary path : D:/LLVM/19.1.1/dist/bin
-- Found LLVM 19.1.3:
-- - CMake module path : D:/LLVM/19.1.3/dist/lib/cmake/llvm
-- - Clang include path : D:/LLVM/19.1.3/dist/include
-- - LLVM Include path : D:/LLVM/19.1.3/dist/include
-- - Binary path : D:/LLVM/19.1.3/dist/bin
-- ---- The below configuring for hipify-clang testing only ----
-- Found Python: C:/Users/TT/AppData/Local/Programs/Python/Python312/python.exe (found version "3.12.7") found components: Interpreter
-- Found lit: C:/Users/TT/AppData/Local/Programs/Python/Python312/Scripts/lit.exe
-- Found FileCheck: D:/LLVM/19.1.1/dist/bin/FileCheck.exe
-- Found Python: C:/Users/TT/AppData/Local/Programs/Python/Python313/python.exe (found version "3.13.0") found components: Interpreter
-- Found lit: C:/Users/TT/AppData/Local/Programs/Python/Python313/Scripts/lit.exe
-- Found FileCheck: D:/LLVM/19.1.3/dist/bin/FileCheck.exe
-- Initial CUDA to configure:
-- - CUDA Toolkit path : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6
-- - CUDA Samples path : C:/ProgramData/NVIDIA Corporation/CUDA Samples/v12.5
-- - cuDNN path : D:/CUDA/cuDNN/9.5.0
-- - cuDNN path : D:/CUDA/cuDNN/9.5.1
-- - CUB path :
-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/include (found version "12.6.68")
-- Found CUDA config:
-- - CUDA Toolkit path : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6
-- - CUDA Samples path : C:/ProgramData/NVIDIA Corporation/CUDA Samples/v12.5
-- - cuDNN path : D:/CUDA/cuDNN/9.5.0
-- - cuDNN path : D:/CUDA/cuDNN/9.5.1
-- - CUB path : C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/include/cub
-- Configuring done (2.1s)
-- Generating done (0.1s)
Expand Down
2 changes: 1 addition & 1 deletion docs/sphinx/requirements.in
View file
Original file line number Diff line number Diff line change
@@ -1 +1 @@
rocm-docs-core==1.8.2
rocm-docs-core==1.8.3
2 changes: 1 addition & 1 deletion docs/sphinx/requirements.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ requests==2.32.2
# via
# pygithub
# sphinx
rocm-docs-core==1.8.2
rocm-docs-core==1.8.3
# via -r requirements.in
smmap==5.0.1
# via gitdb
Expand Down
Loading

0 comments on commit 1b2b520

Please sign in to comment.