Merge branch 'develop' into 3432-improve-simplify_algebra-to-find-mor…

…e-horizontal-fusion-opportunities
ROCm · Dec 30, 2024 · 8bd694e · 8bd694e
2 parents 0d99d23 + 6acc1f9
commit 8bd694e
Show file tree

Hide file tree

Showing 360 changed files with 11,246 additions and 1,555 deletions.
diff --git a/.clang-tidy b/.clang-tidy
@@ -115,3 +115,5 @@ CheckOptions:
     value:           UPPER_CASE
   - key:             readability-identifier-naming.MacroDefinitionPrefix
     value:           MIGRAPHX_
+  - key:             readability-identifier-naming.ConstexprMethodIgnoredRegexp
+    value:           'quiet_NaN|signaling_NaN'
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -595,4 +595,16 @@ jobs:
       #    chmod +x codecov
       #    ./codecov -t ${CODECOV_TOKEN}
       #    echo "Uploaded"
+  misspell:
+    name: misspell
+    runs-on: ubuntu-20.04
+    steps:
+      - name: Check out code.
+        uses: actions/checkout@v4
+      - name: misspell
+        uses: reviewdog/action-misspell@v1
+        with:
+          locale: "US"
+          reporter: github-pr-check
+          level: warning
 
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,6 +3,83 @@
 Full documentation for MIGraphX is available at
 [https://rocmdocs.amd.com/projects/AMDMIGraphX/en/latest/](https://rocmdocs.amd.com/projects/AMDMIGraphX/en/latest/).
 
+## MIGraphX 2.11 for ROCm 6.3.0
+
+### Added
+
+* Initial code to run on Windows
+* Support for gfx120x GPU
+* Support for FP8, and INT4
+* Support for the Log2 internal operator
+* Support for the GCC 14 compiler
+* The BitwiseAnd, Scan, SoftmaxCrossEntropyLoss, GridSample, and NegativeLogLikelihoodLoss ONNX operators
+* The MatMulNBits, QuantizeLinear/DequantizeLinear, GroupQueryAttention, SkipSimplifiedLayerNormalization, and SimpliedLayerNormalization Microsoft Contrib operators
+* Dymamic batch parameter support to OneHot operator
+* Split-K as an optional performance improvement
+* Scripts to validate ONNX models from the ONNX Model Zoo
+* GPU Pooling Kernel
+* --mlir flag to the migraphx-driver program to offload entire module to mlir
+* Fusing split-reduce with MLIR
+* Multiple outputs for the MLIR + Pointwise fusions
+* Pointwise fusions with MLIR across reshape operations
+* MIGRAPHX_MLIR_DUMP environment variable to dump MLIR modules to MXRs
+* The 3 option to MIGRAPHX_TRACE_BENCHMARKING to print the MLIR program for improved debug output
+* MIGRAPHX_ENABLE_HIPBLASLT_GEMM environment variable to call hipBlasLt libaries
+* MIGRAPHX_VERIFY_DUMP_DIFF to improve the debugging of accuracy issues
+* reduce_any and reduce_all options to the Reduce operation via Torch MIGraphX
+* Examples for RNNT, and ControlNet
+
+
+### Changed
+
+* Switched to MLIR's 3D Convolution operator.
+* MLIR is now used for Attention operations by default on gfx942 and newer ASICs.
+* Names and locations for VRM specific libraries have changed.
+* Use random mode for benchmarking GEMMs and convolutions.
+* Python version is now printed with an actual version number.
+
+
+### Removed
+
+* Disabled requirements for MIOpen and rocBlas when running on Windows.
+* Removed inaccuracte warning messages when using exhaustive-tune.
+* Remove the hard coded path in MIGRAPHX_CXX_COMPILER allowing the compiler to be installed in different locations.
+
+
+### Optimized
+
+* Improved:
+    * Infrastructure code to enable better Kernel fusions with all supported data types
+    * Subsequent model compile time by creating a cache for already performant kernels
+    * Use of Attention fusion with models
+    * Performance of the Softmax JIT kernel and of the Pooling opterator
+    * Tuning operations through a new 50ms delay before running the next kernel
+    * Performance of several convolution based models through an optimized NHWC layout
+    * Performance for the FP8 datatype
+    * GPU utilization
+    * Verification tools
+    * Debug prints
+    * Documentation, including gpu-driver utility documentation
+    * Summary section of the migrahx-driver perf command
+* Reduced model compilation time
+* Reordered some compiler passes to allow for more fusions
+* Preloaded tiles into LDS to improve performance of pointwise transposes
+* Exposed the external_data_path property in onnx_options to set the path from onnxruntime
+
+
+### Resolved Issues
+
+* Fixed a bug with gfx1030 that overwrote dpp_reduce.
+* Fixed a bug in 1arg dynamic reshape that created a failure.
+* Fixed a bug with dot_broadcast and inner_broadcast that caused compile failures.
+* Fixed a bug where some configs were failing when using exhaustive-tune.
+* Fixed the ROCM Install Guide URL.
+* Fixed an issue while building a whl package due to an apostrophe.
+* Fixed the BERT Squad example requirements file to support different versions of Python.
+* Fixed a bug that stopped the Vicuna model from compiling.
+* Fixed failures with the verify option of migraphx-driver that would cause the application to exit early.
+
+
 ## MIGraphX 2.10 for ROCm 6.2.0
 
 ### Additions

diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -106,7 +106,7 @@ include(ROCMSetupVersion)
 
 option(BUILD_DEV "Build for development purpose only" OFF)
 
-rocm_setup_version(VERSION 2.11.0)
+rocm_setup_version(VERSION 2.12.0)
 math(EXPR MIGRAPHX_SO_MAJOR_VERSION "(${PROJECT_VERSION_MAJOR} * 1000 * 1000) + (${PROJECT_VERSION_MINOR} * 1000) + ${PROJECT_VERSION_PATCH}")
 set(MIGRAPHX_SO_VERSION ${MIGRAPHX_SO_MAJOR_VERSION}.0)
 
@@ -178,10 +178,8 @@ rocm_enable_clang_tidy(
         -bugprone-easily-swappable-parameters
         -bugprone-implicit-widening-of-multiplication-result
         -bugprone-macro-parentheses
-	-bugprone-multi-level-implicit-pointer-conversion
         -bugprone-signed-char-misuse
         -bugprone-unchecked-optional-access
-	-bugprone-unused-local-non-trivial-variable
         # Disable the aliased reserved identifiers
         -cert-dcl37-c
         -cert-dcl51-cpp

diff --git a/Dockerfile b/Dockerfile
@@ -10,7 +10,7 @@ RUN apt-get update && apt-get install -y software-properties-common gnupg2 --no-
     curl -sL http://repo.radeon.com/rocm/rocm.gpg.key | apt-key add -
 
 # Add rocm repository
-RUN sh -c 'echo deb [arch=amd64 trusted=yes] http://repo.radeon.com/rocm/apt/6.2/ jammy main > /etc/apt/sources.list.d/rocm.list'
+RUN sh -c 'echo deb [arch=amd64 trusted=yes] http://repo.radeon.com/rocm/apt/6.3/ jammy main > /etc/apt/sources.list.d/rocm.list'
 
 # From docs.amd.com for installing rocm. Needed to install properly
 RUN sh -c "echo 'Package: *\nPin: release o=repo.radeon.com\nPin-priority: 600' > /etc/apt/preferences.d/rocm-pin-600"

diff --git a/Jenkinsfile b/Jenkinsfile
@@ -71,7 +71,7 @@ def rocmtestnode(Map conf) {
                     pre()
                     sh "docker pull ${DOCKER_IMAGE}:${env.IMAGE_TAG}"
                     withDockerContainer(image: "${DOCKER_IMAGE}:${env.IMAGE_TAG}", args: "--device=/dev/kfd --device=/dev/dri --group-add video --cap-add SYS_PTRACE -v=/home/jenkins/:/home/jenkins ${docker_args}") {
-                        timeout(time: 2, unit: 'HOURS') {
+                        timeout(time: 4, unit: 'HOURS') {
                             body(cmake_build)
                         }
                     }

diff --git a/codecov.yml b/codecov.yml
@@ -2,3 +2,4 @@ ignore:
   - "test/"
   - "src/driver"
   - "build/" 
+  - "src/netron_output.cpp"
diff --git a/docs/dev/data.rst b/docs/dev/data.rst
@@ -1,3 +1,7 @@
+.. meta::
+  :description: MIGraphX internal data types
+  :keywords: MIGraphX, code base, contribution, developing, data types
+
 Data types
 ==========
 

diff --git a/docs/dev/dev_intro.rst b/docs/dev/dev_intro.rst
@@ -1,3 +1,8 @@
+.. meta::
+  :description: MIGraphX introduction to developing for the code base
+  :keywords: MIGraphX, code base, contribution, developing, introduction, developers
+
+
 Developer Introduction
 ======================
 

diff --git a/docs/dev/env_vars.rst b/docs/dev/env_vars.rst
@@ -1,3 +1,7 @@
+.. meta::
+  :description: MIGraphX internal environment variables
+  :keywords: MIGraphX, code base, contribution, developing, env vars, environment variables
+
 Environment Variables
 =====================
 
@@ -112,9 +116,9 @@ Disables the ``schedule`` pass.
 Set to "1", "enable", "enabled", "yes", or "true" to use.
 Disables the ``fuse_reduce`` pass.
 
-.. envvar:: MIGRAPHX_ENABLE_SPLIT_REDUCE
-Set to "1", "enable", "enabled", "yes", or "true" to use.
-Enable split_reduce.
+.. envvar:: MIGRAPHX_SPLIT_REDUCE_SIZE
+Set to the minimum size of a reduction to do a split reduce. Overrides what
+is set in the backend. Set to -1 to disable split reduce completely.
 
 .. envvar:: MIGRAPHX_ENABLE_NHWC
 

diff --git a/docs/dev/matchers.rst b/docs/dev/matchers.rst
@@ -1,3 +1,7 @@
+.. meta::
+  :description: MIGraphX internal matchers
+  :keywords: MIGraphX, code base, contribution, developing, matchers
+
 Matchers
 ========
 

diff --git a/docs/dev/onnx_operators.rst b/docs/dev/onnx_operators.rst
@@ -1,22 +1,24 @@
+.. meta::
+  :description: MIGraphX supported ONNX operators
+  :keywords: MIGraphX, code base, contribution, developing, ONNX operators
+
 Supported ONNX Operators
 ========================
 
 MIGraphX supports operators up to Opset 19. Latest information of ONNX
-operators can be found
-`here <https://github.com/onnx/onnx/blob/master/docs/Operators.md>`__
+operators can be found in `the ONNX GitHub repository <https://github.com/onnx/onnx/blob/master/docs/Operators.md>`_.
 
-MIGraphX supports the following ONNX data types: BOOL, UINT8, UINT16,
-UINT32, UINT64, INT8, INT16, INT32, INT64, FLOAT8, FLOAT16, FLOAT32,
-DOUBLE
+MIGraphX supports the following ONNX data types: BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FLOAT8, FLOAT16, FLOAT32, and DOUBLE
 
-   NOTE: FP8 support is only for E4M3FNUZ, see
-   `here <https://onnx.ai/onnx/technical/float8.html>`__
+   .. Note::
+
+      FP8 support is only for E4M3FNUZ, see `Float stored in 8 bits <https://onnx.ai/onnx/technical/float8.html>`_ in the ONNX documentation.
 
 See below for the support matrix of ONNX operators in MIGraphX.
 
-   NOTE: Supported Types are from ONNX specification. An operator might
-   support more datatypes (e.g. integer type for float operator) than
-   listed.
+   .. Note:: 
+
+      The listed supported types are taken from the ONNX specification. An operator might support other additional datatypes.
 
 Operator Support Matrix
 -----------------------

diff --git a/docs/dev/operators.rst b/docs/dev/operators.rst
@@ -1,3 +1,7 @@
+.. meta::
+  :description: MIGraphX internal operators
+  :keywords: MIGraphX, code base, contribution, developing, operators
+
 Operators
 =========
 

diff --git a/docs/dev/pass.rst b/docs/dev/pass.rst
@@ -1,3 +1,7 @@
+.. meta::
+  :description: MIGraphX internal passes
+  :keywords: MIGraphX, code base, contribution, developing, passes
+
 Passes
 ======
 

diff --git a/docs/dev/program.rst b/docs/dev/program.rst
@@ -1,3 +1,7 @@
+.. meta::
+  :description: MIGraphX program
+  :keywords: MIGraphX, code base, contribution, developing, program
+
 Program
 =======
 

diff --git a/docs/dev/quantization.rst b/docs/dev/quantization.rst
@@ -1,3 +1,7 @@
+.. meta::
+  :description: MIGraphX internal quantization
+  :keywords: MIGraphX, code base, contribution, developing, quantization
+
 Quantization
 ============
 

diff --git a/docs/dev/targets.rst b/docs/dev/targets.rst
@@ -1,3 +1,7 @@
+.. meta::
+  :description: MIGraphX targets
+  :keywords: MIGraphX, code base, contribution, developing, targets
+
 Targets
 =======
 

diff --git a/docs/dev/tools.rst b/docs/dev/tools.rst
@@ -1,3 +1,7 @@
+.. meta::
+  :description: MIGraphX tools
+  :keywords: MIGraphX, code base, contribution, developing, tooks, knobs
+
 .. _tools:
 
 Tools

diff --git a/docs/dev/triage-rocmlir.rst b/docs/dev/triage-rocmlir.rst
@@ -1,3 +1,7 @@
+.. meta::
+  :description: Issue Triaging Guide for suspected issues
+  :keywords: MIGraphX, rocMLIR, issues, pipeline, compilation, bug, code base, kernel, contribution, developing
+
 Issue Triaging Guide for suspected rocMLIR issue
 ================================================
 

diff --git a/docs/index.rst b/docs/index.rst
@@ -25,21 +25,22 @@ The MIGraphX public repository is located at `https://github.com/ROCm/AMDMIGraph
 
   .. grid-item-card:: Install
 
-    * :doc:`Installing MIGraphX <./install/installing_with_package>`
+    * :doc:`Installing MIGraphX with the package installer <./install/installing_with_package>`
+    * :doc:`Building and installing MIGraphX from source code <./install/building_migraphx>`
 
-  .. grid-item-card:: Using the MIGraphX API
+  .. grid-item-card:: Reference
 
     * :ref:`cpp-api-reference`
     * :ref:`python-api-reference`
     * :ref:`migraphx-driver`
+    * :doc:`Supported ONNX Operators <./dev/onnx_operators>`
 
   .. grid-item-card:: Contributing to the MIGraphX code base
 
-    * :doc:`Building MIGraphX <./install/building_migraphx>`
     * :doc:`Developing for MIGraphX <./dev/contributing-to-migraphx>` 
 
 To contribute to the documentation refer to
-`Contribute to ROCm documentation <https://rocm.docs.amd.com/en/latest/contribute/contributing.html>`_.
+`Contributing to ROCm <https://rocm.docs.amd.com/en/latest/contribute/contributing.html>`_.
 
 Licensing information can be found on the
 `Licensing <https://rocm.docs.amd.com/en/latest/about/license.html>`_ page.
diff --git a/docs/install/build_and_install_with_cmake.rst b/docs/install/build_and_install_with_cmake.rst