[TRANSFORMATIONS][GPU] Add GroupNormalization fusion to common optimizations #28387

jhajducz · 2025-01-11T02:03:54Z

Details:

Added GroupNormalization fusion pass that can handle pattern observed in many customer models that were exported via ONNX in a way that uses InstanceNormalization as a proxy for GroupNormalization. It covers also more traditional cases without additional instance norm related parameters.
Per suggestion from @vladimir-paramuzov, for now enabled GroupNormalization fusion only for GPU plugin. Once it will be verified that it doesn't cause regressions in other backends, we can enable it for them as well.

Tickets:

160436

mlukasze · 2025-01-13T05:52:12Z

build_jenkins

t-jankowski

Lgtm regarding Core part.

src/common/transformations/tests/common_optimizations/group_normalization_fusion_tests.cpp

src/core/include/openvino/core/type/element_type.hpp

praasz · 2025-01-13T11:14:44Z

build_jenkins

...mmon/transformations/src/transformations/common_optimizations/group_normalization_fusion.cpp

vladimir-paramuzov · 2025-01-15T04:54:29Z

...mmon/transformations/src/transformations/common_optimizations/group_normalization_fusion.cpp

+        auto input = pattern_map.at(input_m);
+        auto input_ps = input.get_partial_shape();
+
+        auto T = input.get_element_type();


the high level approach for matcher passes is that we split what to replace (pattern) and how to replace (callback). So ideally, all applicability checks should be a part of pattern, including this type/shapes/ranks checks. It's not always possible (for instance, when we need to cross-check multiple Ops), but for op attributes or in/out ports checks you can just use predicates mechanism. You can find few examples here: https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/plugin/transformations/transpose_fusion.cpp#L88-L98

I've changed it to use predicates for things that don't depend on other nodes, but as you pointed out, I had to still leave some cross-checks between different ops inside callback. Can you check if it looks good to you now?

src/plugins/intel_gpu/src/plugin/transformations_pipeline.cpp

.../transformations/include/transformations/common_optimizations/group_normalization_fusion.hpp

...mmon/transformations/src/transformations/common_optimizations/group_normalization_fusion.cpp

src/common/transformations/tests/common_optimizations/group_normalization_fusion_tests.cpp

praasz · 2025-01-15T06:56:05Z

src/common/transformations/tests/common_optimizations/group_normalization_fusion_tests.cpp

+#include "transformations/init_node_info.hpp"
+
+using namespace testing;
+using namespace ov;


Consider wrap test in namespace

Suggested change

using namespace ov;

namespace ov::test{

While I appreciate your suggestion, I prefer to keep it consistent with other tests for common optimizations.

src/common/transformations/tests/common_optimizations/group_normalization_fusion_tests.cpp

praasz · 2025-01-15T07:19:59Z

...mmon/transformations/src/transformations/common_optimizations/group_normalization_fusion.cpp

+        // we need to be able to cast mvn to MVN layer type
+        // in order to read actual epsilon value
+        auto mvn_out = pattern_map.at(mvn_m);
+        auto mvn = std::dynamic_pointer_cast<ov::op::v6::MVN>(mvn_out.get_node_shared_ptr());


Suggested change

auto mvn = std::dynamic_pointer_cast<ov::op::v6::MVN>(mvn_out.get_node_shared_ptr());

auto mvn = ov::as_type_ptr<ov::op::v6::MVN>(mvn_out.get_node_shared_ptr());

Use OV cast,
Or use as_type<op::v6::MVN*>(mvn_out.get_node()) to not create shared pointers as it looks not required for mvn.,

the mvn can be null pointer or is validated before and here is not possible?

Thanks, I've changed it to OV cast. My understanding is that there is no need for additional check, as it is set in the pattern as node of type "ov::op::v6::MVN" already.

src/common/transformations/tests/common_optimizations/group_normalization_fusion_tests.cpp

praasz · 2025-01-15T07:28:05Z

.../transformations/include/transformations/common_optimizations/group_normalization_fusion.hpp

+namespace ov {
+namespace pass {


Optional:

Suggested change

namespace ov {

namespace pass {

namespace ov::pass {

While I appreciate your suggestion, I prefer to keep it consistent with header files for other common optimizations.

.../transformations/include/transformations/common_optimizations/group_normalization_fusion.hpp

This reverts commit 217bcdc.

…GroupNormalizationFusion tests

…izationFusion tests

…lizationFusion tests

…sformations pipeline

…es in GroupNormalizationFusion pass

github-actions bot added category: Core OpenVINO Core (aka ngraph) category: GPU OpenVINO GPU plugin category: transformations OpenVINO Runtime library - Transformations category: CPP API OpenVINO CPP API bindings labels Jan 11, 2025

sys-openvino-ci added the ExternalIntelPR External contributor from Intel label Jan 11, 2025

jhajducz force-pushed the private/jhajducz/gpu_gn_fusion branch from 3a67c81 to ba87e35 Compare January 12, 2025 17:37

jhajducz marked this pull request as ready for review January 13, 2025 00:38

jhajducz requested review from a team as code owners January 13, 2025 00:38

jhajducz requested review from itikhono and removed request for a team January 13, 2025 00:38

t-jankowski approved these changes Jan 13, 2025

View reviewed changes

src/common/transformations/tests/common_optimizations/group_normalization_fusion_tests.cpp Show resolved Hide resolved

jhajducz force-pushed the private/jhajducz/gpu_gn_fusion branch from ba87e35 to 3bd623d Compare January 13, 2025 10:49

praasz requested changes Jan 13, 2025

View reviewed changes

src/core/include/openvino/core/type/element_type.hpp Outdated Show resolved Hide resolved

jhajducz force-pushed the private/jhajducz/gpu_gn_fusion branch from 3bd623d to d69391b Compare January 14, 2025 13:05

github-actions bot removed category: Core OpenVINO Core (aka ngraph) category: CPP API OpenVINO CPP API bindings labels Jan 14, 2025

jhajducz requested a review from praasz January 14, 2025 15:39

jhajducz force-pushed the private/jhajducz/gpu_gn_fusion branch from d69391b to b1ac67d Compare January 14, 2025 20:29

vladimir-paramuzov reviewed Jan 15, 2025

View reviewed changes

praasz reviewed Jan 15, 2025

View reviewed changes

t-jankowski requested changes Jan 15, 2025

View reviewed changes

.../transformations/include/transformations/common_optimizations/group_normalization_fusion.hpp Outdated Show resolved Hide resolved

jhajducz force-pushed the private/jhajducz/gpu_gn_fusion branch 2 times, most recently from 89fe905 to 9cf1017 Compare January 16, 2025 17:32

jhajducz added 2 commits January 16, 2025 21:23

Make conversion of basic data type to element type a constexpr

53f98ee

Add GroupNormalization fusion to common optimizations

88e9bb6

jhajducz added 19 commits January 16, 2025 21:23

Add GroupNormalization fusion tests

31d2c5c

Enable GroupNormalization fusion pass in GPU plugin

05257ae

Update copyright notice

80c717f

Revert "Make conversion of basic data type to element type a constexpr"

b970670

This reverts commit 217bcdc.

Refactor GroupNormalizationFusion tests to avoid changes in core API

19ef1dc

Remove GPU plugin specific GroupNormComposition pass

2219ed9

Fix RTTI macro in GroupNormalizationFusion header file

ee0b47d

Override TestBody() method in GroupNormalizationFusionTestsFixture

9189fe8

Explain meaning of GroupNormalizationFusion tests parameters

a240143

Require providing correct group norm gamma & beta shapes in positive …

ee87f86

…GroupNormalizationFusion tests

Use dedicated Constant ctor to create scalar constants in GroupNormal…

daa7bbf

…izationFusion tests

Avoid Shape->PartialShape conversion for in/out tensors in GroupNorma…

fb7d676

…lizationFusion tests

Use global testing namespace in GroupNormalizationFusion tests

cb8ff03

Another update of copyright notice

386578c

Use const references where possible in GroupNormalizationFusion pass

965fe39

Move GroupNormalizationFusion after MVNFusion pass in GPU plugin tran…

17572e6

…sformations pipeline

Use OV ptr cast for MVN in GroupNormalizationFusion pass

f063b8c

Add 5d and 6d cases to GroupNormalizationFusion tests + fix formatting

e3a344c

Use predicates for type & shape checks that don't depend on other nod…

8382572

…es in GroupNormalizationFusion pass

jhajducz force-pushed the private/jhajducz/gpu_gn_fusion branch from 9cf1017 to 8382572 Compare January 16, 2025 20:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TRANSFORMATIONS][GPU] Add GroupNormalization fusion to common optimizations #28387

[TRANSFORMATIONS][GPU] Add GroupNormalization fusion to common optimizations #28387

jhajducz commented Jan 11, 2025 •

edited

Loading

mlukasze commented Jan 13, 2025

t-jankowski left a comment

praasz commented Jan 13, 2025

vladimir-paramuzov Jan 15, 2025 •

edited

Loading

jhajducz Jan 16, 2025

praasz Jan 15, 2025

jhajducz Jan 16, 2025

praasz Jan 15, 2025

jhajducz Jan 16, 2025

praasz Jan 15, 2025

jhajducz Jan 16, 2025

	auto mvn = std::dynamic_pointer_cast<ov::op::v6::MVN>(mvn_out.get_node_shared_ptr());
	auto mvn = ov::as_type_ptr<ov::op::v6::MVN>(mvn_out.get_node_shared_ptr());

[TRANSFORMATIONS][GPU] Add GroupNormalization fusion to common optimizations #28387

Are you sure you want to change the base?

[TRANSFORMATIONS][GPU] Add GroupNormalization fusion to common optimizations #28387

Conversation

jhajducz commented Jan 11, 2025 • edited Loading

Details:

Tickets:

mlukasze commented Jan 13, 2025

t-jankowski left a comment

Choose a reason for hiding this comment

praasz commented Jan 13, 2025

vladimir-paramuzov Jan 15, 2025 • edited Loading

Choose a reason for hiding this comment

jhajducz Jan 16, 2025

Choose a reason for hiding this comment

praasz Jan 15, 2025

Choose a reason for hiding this comment

jhajducz Jan 16, 2025

Choose a reason for hiding this comment

praasz Jan 15, 2025

Choose a reason for hiding this comment

jhajducz Jan 16, 2025

Choose a reason for hiding this comment

praasz Jan 15, 2025

Choose a reason for hiding this comment

jhajducz Jan 16, 2025

Choose a reason for hiding this comment

jhajducz commented Jan 11, 2025 •

edited

Loading

vladimir-paramuzov Jan 15, 2025 •

edited

Loading