From ef1298d19addc7221e7bc7cefce0a43e19bd4849 Mon Sep 17 00:00:00 2001 From: "Shen, Wanglei" Date: Mon, 30 Dec 2024 08:43:08 +0800 Subject: [PATCH] update doc for using one socket with latency hint instead of one numa node --- ...performance-hint-and-thread-scheduling.rst | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/articles_en/openvino-workflow/running-inference/inference-devices-and-modes/cpu-device/performance-hint-and-thread-scheduling.rst b/docs/articles_en/openvino-workflow/running-inference/inference-devices-and-modes/cpu-device/performance-hint-and-thread-scheduling.rst index 46b541d84d4035..6cc211116d1199 100644 --- a/docs/articles_en/openvino-workflow/running-inference/inference-devices-and-modes/cpu-device/performance-hint-and-thread-scheduling.rst +++ b/docs/articles_en/openvino-workflow/running-inference/inference-devices-and-modes/cpu-device/performance-hint-and-thread-scheduling.rst @@ -63,19 +63,19 @@ the model precision and the ratio of P-cores and E-cores. Then the default settings for low-level performance properties on Windows and Linux are as follows: -+--------------------------------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------+ -| Property | Windows | Linux | -+======================================+=======================================================================+=======================================================================+ -| ``ov::num_streams`` | 1 | 1 | -+--------------------------------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------+ -| ``ov::inference_num_threads`` | is equal to the number of P-cores or P-cores+E-cores on one numa node | is equal to the number of P-cores or P-cores+E-cores on one numa node | -+--------------------------------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------+ -| ``ov::hint::scheduling_core_type`` | :ref:`Core Type Table of Latency Hint ` | :ref:`Core Type Table of Latency Hint ` | -+--------------------------------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------+ -| ``ov::hint::enable_hyper_threading`` | No | No | -+--------------------------------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------+ -| ``ov::hint::enable_cpu_pinning`` | No / Not Supported | Yes except using P-cores and E-cores together | -+--------------------------------------+-----------------------------------------------------------------------+-----------------------------------------------------------------------+ ++--------------------------------------+--------------------------------------------------------------------+--------------------------------------------------------------------+ +| Property | Windows | Linux | ++======================================+====================================================================+====================================================================+ +| ``ov::num_streams`` | 1 | 1 | ++--------------------------------------+--------------------------------------------------------------------+--------------------------------------------------------------------+ +| ``ov::inference_num_threads`` | is equal to the number of P-cores or P-cores+E-cores on one socket | is equal to the number of P-cores or P-cores+E-cores on one socket | ++--------------------------------------+--------------------------------------------------------------------+--------------------------------------------------------------------+ +| ``ov::hint::scheduling_core_type`` | :ref:`Core Type Table of Latency Hint ` | :ref:`Core Type Table of Latency Hint ` | ++--------------------------------------+--------------------------------------------------------------------+--------------------------------------------------------------------+ +| ``ov::hint::enable_hyper_threading`` | No | No | ++--------------------------------------+--------------------------------------------------------------------+--------------------------------------------------------------------+ +| ``ov::hint::enable_cpu_pinning`` | No / Not Supported | Yes except using P-cores and E-cores together | ++--------------------------------------+--------------------------------------------------------------------+--------------------------------------------------------------------+ .. note:: @@ -96,7 +96,7 @@ Then the default settings for low-level performance properties on Windows and Li Starting from 5th Gen Intel Xeon Processors, new microarchitecture enabled new sub-NUMA clusters feature. A sub-NUMA cluster (SNC) can create two or more localization domains (numa nodes) within a socket by BIOS configuration. - By default OpenVINO with latency hint uses single NUMA node for inference. Although such + By default OpenVINO with latency hint uses single socket for inference. Although such behavior allows to achive best performance for most of the models, there might be corner cases which require manual tuning of ``ov::num_streams`` and ``ov::hint::enable_hyper_threading parameters``. Please find more detail about `Sub-NUMA Clustering `__