add new feature docs (#1286)

add 1.13 new feature docs

add hypertune strategy img

update to intel_extension_for_pytorch.cpu.hypertune

modify the docs of graph capture and recipe tuning

modify recipe tuning doc

fix typo in ipex.quantization.convert api doc

update installation index html url and add link for ipex xpu

align release notes to 1.12.300

change auto_opt into auto_ipex

remove the name of IPEX

adjust xpu hp url and int8 feature docs url

list new feature abstracts to features.rst

add keywords of gpu for seo

fine tune

fine tune

fine tune

fine tune

Author: jingxu10

docker/Dockerfile.conda

@@ -41,10 +41,10 @@ RUN curl -fsSL -v -o ~/miniconda.sh -O  https://repo.anaconda.com/miniconda/Mini
     /opt/conda/bin/conda clean -ya
 
 FROM dev-base AS build
-ARG IPEX_VERSION=v1.12.100
-ARG PYTORCH_VERSION=v1.12.0
-ARG TORCHVISION_VERSION=0.13.0+cpu
-ARG TORCHAUDIO_VERSION=0.12.0+cpu
+ARG IPEX_VERSION=v1.13.0
+ARG PYTORCH_VERSION=v1.13.0
+ARG TORCHVISION_VERSION=0.14.0+cpu
+ARG TORCHAUDIO_VERSION=0.13.0+cpu
 COPY --from=conda /opt/conda /opt/conda
 RUN --mount=type=cache,target=/opt/ccache \
     python -m pip install --no-cache-dir torch==${PYTORCH_VERSION}+cpu torchvision==${TORCHVISION_VERSION} torchaudio==${TORCHAUDIO_VERSION} -f https://download.pytorch.org/whl/torch_stable.html && \

docker/Dockerfile.pip

@@ -30,10 +30,10 @@ RUN ${PYTHON} -m pip --no-cache-dir install --upgrade \
 # Some TF tools expect a "python" binary
 RUN ln -s $(which ${PYTHON}) /usr/local/bin/python
 
-ARG IPEX_VERSION=1.12.100
-ARG PYTORCH_VERSION=1.12.0+cpu
-ARG TORCHAUDIO_VERSION=0.12.0
-ARG TORCHVISION_VERSION=0.13.0+cpu
+ARG IPEX_VERSION=1.13.0
+ARG PYTORCH_VERSION=1.13.0+cpu
+ARG TORCHAUDIO_VERSION=0.13.0
+ARG TORCHVISION_VERSION=0.14.0+cpu
 ARG TORCH_CPU_URL=https://download.pytorch.org/whl/cpu/torch_stable.html
 ARG IPEX_URL=https://software.intel.com/ipex-whl-stable
 

docs/index.rst

@@ -1,6 +1,6 @@
 .. meta::
    :description: This website introduces Intel® Extension for PyTorch*
-   :keywords: Intel optimization, PyTorch, Intel® Extension for PyTorch*
+   :keywords: Intel optimization, PyTorch, Intel® Extension for PyTorch*, GPU, discrete GPU, Intel discrete GPU
 
 Welcome to Intel® Extension for PyTorch* Documentation
 ######################################################
@@ -11,19 +11,32 @@ Intel® Extension for PyTorch* provides optimizations for both eager mode and gr
 
 The extension can be loaded as a Python module for Python programs or linked as a C++ library for C++ programs. In Python scripts users can enable it dynamically by importing `intel_extension_for_pytorch`.
 
-**Note**: Check `here <https://intel.github.io/intel-extension-for-pytorch/xpu/latest/>`_ for detailed tutorials of Intel® Extension for PyTorch* for Intel® GPUs. Source code are available at the `xpu-master branch <https://github.com/intel/intel-extension-for-pytorch/tree/xpu-master>`_.
+-------------------------------------
 
 Intel® Extension for PyTorch* for CPU is structured as shown in the following figure:
 
-.. figure:: ../images/intel_extension_for_pytorch_structure.png
+.. figure:: ../images/intel_extension_for_pytorch_structure_cpu.png
   :width: 800
   :align: center
-  :alt: Structure of Intel® Extension for PyTorch*
+  :alt: Structure of Intel® Extension for PyTorch* for CPU
 
 
-PyTorch components are depicted with white boxes while Intel Extensions are with blue boxes. Extra performance of the extension is delivered via both custom addons and overriding existing PyTorch components. In eager mode, the PyTorch frontend is extended with custom Python modules (such as fusion modules), optimal optimizers and INT8 quantization API. Further performance boosting is available by converting the eager-mode model into graph mode via the extended graph fusion passes. Intel® Extension for PyTorch* dispatches the operators into their underlying kernels automatically based on ISA that it detects and leverages vectorization and matrix acceleration units available in Intel hardware, as much as possible. oneDNN library is used for computation intensive operations. Intel Extension for PyTorch runtime extension brings better efficiency with finer-grained thread runtime control and weight sharing.
+PyTorch components are depicted with white boxes while Intel Extensions are with blue boxes. Extra performance of the extension is delivered via both custom addons and overriding existing PyTorch components. In eager mode, the PyTorch frontend is extended with custom Python modules (such as fusion modules), optimal optimizers and INT8 quantization API. Further performance boost is available by converting the eager-mode model into graph mode via the extended graph fusion passes. Intel® Extension for PyTorch* dispatches the operators into their underlying kernels automatically based on ISA that it detects and leverages vectorization and matrix acceleration units available in Intel hardware, as much as possible. oneDNN library is used for computation intensive operations. Intel Extension for PyTorch runtime extension brings better efficiency with finer-grained thread runtime control and weight sharing.
 
-Intel® Extension for PyTorch* has been released as an open–source project at `Github <https://github.com/intel/intel-extension-for-pytorch>`_.
+Intel® Extension for PyTorch* for CPU has been released as an open–source project at `Github master branch <https://github.com/intel/intel-extension-for-pytorch/tree/master>`_. Check `CPU tutorial <https://intel.github.io/intel-extension-for-pytorch/cpu/latest/>`_ for detailed information of Intel® Extension for PyTorch* for Intel® CPUs.
+
+-------------------------------------
+
+Intel® Extension for PyTorch* for GPU is structured as shown in the following figure:
+
+.. figure:: ../images/intel_extension_for_pytorch_structure_gpu.svg
+  :width: 800
+  :align: center
+  :alt: Architecture of Intel® Extension for PyTorch* for GPU
+
+Intel® Extension for PyTorch* for GPU utilizes the `DPC++ <https://github.com/intel/llvm#oneapi-dpc-compiler>`_ compiler that supports the latest `SYCL* <https://registry.khronos.org/SYCL/specs/sycl-2020/html/sycl-2020.html>`_ standard and also a number of extensions to the SYCL* standard, which can be found in the `sycl/doc/extensions <https://github.com/intel/llvm/tree/sycl/sycl/doc/extensions>`_ directory. Intel® Extension for PyTorch* also integrates `oneDNN <https://github.com/oneapi-src/oneDNN>`_ and `oneMKL <https://github.com/oneapi-src/oneMKL>`_ libraries and provides kernels based on that. The oneDNN library is used for computation intensive operations. The oneMKL library is used for fundamental mathematical operations.
+
+Intel® Extension for PyTorch* for GPU has been released as an open–source project on `GitHub xpu-master branch <https://github.com/intel/intel-extension-for-pytorch/tree/xpu-master>`_. Check `GPU tutorial <https://intel.github.io/intel-extension-for-pytorch/xpu/latest/>`_ for detailed information of Intel® Extension for PyTorch* for Intel® GPUs.
 
 .. toctree::
    :hidden:

docs/tutorials/api_doc.rst

@@ -16,6 +16,10 @@ Quantization
 .. autofunction:: prepare
 .. autofunction:: convert
 
+Experimental API, introduction is avaiable at `feature page <./features/int8_recipe_tuning_api.md>`_.
+
+.. autofunction:: autotune
+
 CPU Runtime
 ***********
 

docs/tutorials/examples.md

@@ -1,6 +1,8 @@
 Examples
 ========
 
+**_NOTE:_** Check individual feature page for examples of feature usage. All features are listed in the [feature page](./features.rst).
+
 ## Training
 
 ### Single-instance Training
@@ -611,4 +613,4 @@ $ ldd example-app
 
 ## Model Zoo
 
-Use cases that had already been optimized by Intel engineers are available at [Model Zoo for Intel® Architecture](https://github.com/IntelAI/models/tree/pytorch-r1.12-models). A bunch of PyTorch use cases for benchmarking are also available on the [GitHub page](https://github.com/IntelAI/models/tree/pytorch-r1.12-models/benchmarks#pytorch-use-cases). You can get performance benefits out-of-box by simply running scipts in the Model Zoo.
+Use cases that had already been optimized by Intel engineers are available at [Model Zoo for Intel® Architecture](https://github.com/IntelAI/models/tree/pytorch-r1.13-models). A bunch of PyTorch use cases for benchmarking are also available on the [GitHub page](https://github.com/IntelAI/models/tree/pytorch-r1.13-models/benchmarks#pytorch-use-cases). You can get performance benefits out-of-box by simply running scipts in the Model Zoo.

docs/tutorials/features.rst

@@ -31,30 +31,18 @@ For more detailed information, check `ISA Dynamic Dispatching <features/isa_dyna
 
    features/isa_dynamic_dispatch
 
-Channels Last
--------------
-
-Compared with the default NCHW memory format, using channels_last (NHWC) memory format could further accelerate convolutional neural networks. In Intel® Extension for PyTorch\*, NHWC memory format has been enabled for most key CPU operators, though not all of them have been accepted and merged into the PyTorch master branch yet.
-
-For more detailed information, check `Channels Last <features/nhwc.md>`_.
-
-.. toctree::
-   :hidden:
-   :maxdepth: 1
-
-   features/nhwc
-
 Auto Channels Last
 ------------------
 
-Intel® Extension for PyTorch* automatically converts the model to channels last memory format by default when users optimize their model with ``ipex.optimize(model)``. 
+Comparing to the default NCHW memory format, using channels_last (NHWC) memory format could further accelerate convolutional neural networks. In Intel® Extension for PyTorch*, NHWC memory format has been enabled for most key CPU operators. More detailed information is available at `Channels Last <features/nhwc.md>`_.
 
-For more detailed information, check `Auto Channels Last <features/auto_channels_last.md>`_.
+Intel® Extension for PyTorch* automatically converts a model to channels last memory format when users optimize the model with `ipex.optimize(model)`. With this feature users won't need to manually apply `model=model.to(memory_format=torch.channels_last)` any more. More detailed information is available at `Auto Channels Last <features/auto_channels_last.md>`_.
 
 .. toctree::
    :hidden:
    :maxdepth: 1
 
+   features/nhwc
    features/auto_channels_last
 
 Auto Mixed Precision (AMP)
@@ -89,18 +77,21 @@ Compared to eager mode, graph mode in PyTorch normally yields better performance
 Operator Optimization
 ---------------------
 
-Intel® Extension for PyTorch\* also optimizes operators and implements several customized operators for performance boosts. A few ATen operators are replaced by their optimized counterparts in Intel® Extension for PyTorch\* via the ATen registration mechanism. Some customized operators are implemented for several popular topologies. For instance, ROIAlign and NMS are defined in Mask R-CNN. To improve performance of these topologies, Intel® Extension for PyTorch\* also optimized these customized operators.
+Intel® Extension for PyTorch* also optimizes operators and implements several customized operators for performance boosts. A few ATen operators are replaced by their optimized counterparts in Intel® Extension for PyTorch* via the ATen registration mechanism. Some customized operators are implemented for several popular topologies. For instance, ROIAlign and NMS are defined in Mask R-CNN. To improve performance of these topologies, Intel® Extension for PyTorch* also optimized these customized operators.
 
 .. currentmodule:: intel_extension_for_pytorch.nn
 .. autoclass:: FrozenBatchNorm2d
 
 .. currentmodule:: intel_extension_for_pytorch.nn.functional
 .. autofunction:: interaction
 
+**Auto kernel selection** is a feature that enables users to tune for better performance with GEMM operations. It is provided as parameter –auto_kernel_selection, with boolean value, of the ipex.optimize() function. By default, the GEMM kernel is computed with oneMKL primitives. However, under certain circumstances oneDNN primitives run faster. Users are able to set –auto_kernel_selection to True to run GEMM kernels with oneDNN primitives.” -> "We aims to provide good default performance by leveraging the best of math libraries and enabled weights_prepack, and it has been verified with broad set of models. If you would like to try other alternatives, you can use auto_kernel_selection toggle in ipex.optimize to switch, and you can diesable weights_preack in ipex.optimize if you are concerning the memory footprint more than performance gain. However in majority cases, keeping default is what we recommend.
+
 Optimizer Optimization
 ----------------------
 
-Optimizers are one of key parts of the training workloads. Intel® Extension for PyTorch\* brings two types of optimizations to optimizers:
+Optimizers are one of key parts of the training workloads. Intel® Extension for PyTorch* brings two types of optimizations to optimizers:
+
 1.	Operator fusion for the computation in the optimizers.
 2.	SplitSGD for BF16 training, which reduces the memory footprint of the master weights by half.
 
@@ -114,9 +105,8 @@ For more detailed information, check `Optimizer Fusion <features/optimizer_fusio
    features/optimizer_fusion
    features/split_sgd
 
-
 Runtime Extension
---------------------------------
+-----------------
 
 Intel® Extension for PyTorch* Runtime Extension provides PyTorch frontend APIs for users to get finer-grained control of the thread runtime and provides:
 
@@ -135,14 +125,54 @@ For more detailed information, check `Runtime Extension <features/runtime_extens
    features/runtime_extension
 
 INT8 Quantization
---------------------------------
+-----------------
+
+Intel® Extension for PyTorch* provides built-in quantization recipes to deliver good statistical accuracy for most popular DL workloads including CNN, NLP and recommendation models. On top of that, if users would like to tune for a higher accuracy than what the default recipe provides, a recipe tuning API powered by Intel® Neural Compressor is provided for users to try.
+
+Check more detailed information for `INT8 Quantization <features/int8_overview.md>`_ and `INT8 recipe tuning API guide (Experimental, *NEW feature in 1.13.0*) <features/int8_recipe_tuning_api.md>`_.
+
+.. toctree::
+   :hidden:
+   :maxdepth: 1
+
+   features/int8_overview
+   features/int8_recipe_tuning_api
+
+Codeless Optimization (Experimental, *NEW feature in 1.13.0*)
+-------------------------------------------------------------
+
+This feature enables users to get performance benefits from Intel® Extension for PyTorch* without changing Python scripts. It hopefully eases the usage and has been verified working well with broad scope of models, though in few cases there could be small overhead comparing to applying optimizations with Intel® Extension for PyTorch* APIs.
+
+For more detailed information, check `Codeless Optimization <features/codeless_optimization.md>`_.
+
+.. toctree::
+   :hidden:
+   :maxdepth: 1
+
+   features/codeless_optimization.md
+
+Graph Capture (Experimental, *NEW feature in 1.13.0*)
+-----------------------------------------------------
+
+Since graph mode is key for deployment performance, this feature automatically captures graphs based on set of technologies that PyTorch supports, such as TorchScript and TorchDynamo. Users won't need to learn and try different PyTorch APIs to capture graphs, instead, they can turn on a new boolean flag `--graph_mode` (default off) in `ipex.optimize` to get the best of graph optimization.
+
+For more detailed information, check `Graph Capture <features/graph_capture.md>`_.
+
+.. toctree::
+   :hidden:
+   :maxdepth: 1
+
+   features/graph_capture
+
+HyperTune (Experimental, *NEW feature in 1.13.0*)
+-------------------------------------------------
 
-Intel® Extension for PyTorch* has built-in quantization recipes to deliver good statistical accuracy for most popular DL workloads including CNN, NLP and recommendation models.
+HyperTune is an experimental feature to perform hyperparameter/execution configuration searching. The searching is used in various areas such as optimization of hyperparameters of deep learning models. The searching is extremely useful in real situations when the number of hyperparameters, including configuration of script execution, and their search spaces are huge that manually tuning these hyperparameters/configuration is impractical and time consuming. Hypertune automates this process of execution configuration searching for the `launcher <performance_tuning/launch_script.md>`_ and Intel® Extension for PyTorch*.
 
-Check more detailed information for `INT8 Quantization <features/int8.md>`_.
+For more detailed information, check `HyperTune <features/hypertune.md>`_.
 
 .. toctree::
    :hidden:
    :maxdepth: 1
 
-   features/int8
+   features/hypertune