Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OP][CPU] Fix SliceScatter issues with non-constant slice params #27482

Draft
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

mmikolajcz
Copy link
Contributor

Details:

  • SliceScatter could experience failures caused by not calling prepareParams() depending on input and execution type
  • Fix issue in calling get_raw_data in shape_inference on empty tensors

Tickets:

@mmikolajcz mmikolajcz requested review from a team as code owners November 8, 2024 13:51
@github-actions github-actions bot added category: Core OpenVINO Core (aka ngraph) category: CPU OpenVINO CPU plugin category: TEMPLATE OpenVINO Template plugin labels Nov 8, 2024
@mmikolajcz mmikolajcz marked this pull request as draft November 8, 2024 16:57
@mmikolajcz mmikolajcz marked this pull request as ready for review November 19, 2024 10:18
src/plugins/intel_cpu/src/nodes/strided_slice.cpp Outdated Show resolved Hide resolved
Comment on lines 330 to 333
if (!execPtr && !Node::isDynamic) {
// SliceScatter due to not having data dependency on shape may not call prepareParams when start/stop/step values are non-constant in Static execution.
StridedSlice::prepareParams();
} else if (!execPtr) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same logic could be implemented without duplication of the !execPtr check.
As the comment is related to the SliceScatter op, shouldn't the attrs.isSliceScatterOp be checked as well?
Also is there a chance that the prepareParams will not initialize the execPtr and it will be still a nullptr after?

Suggested change
if (!execPtr && !Node::isDynamic) {
// SliceScatter due to not having data dependency on shape may not call prepareParams when start/stop/step values are non-constant in Static execution.
StridedSlice::prepareParams();
} else if (!execPtr) {
if (!execPtr) {
if (attrs.isSliceScatterOp && !isDynamicNode()) {
// SliceScatter due to not having data dependency on shape may not call prepareParams when start/stop/step values are non-constant in Static execution.
StridedSlice::prepareParams();
} else {
OPENVINO_THROW(errorPrefix, "doesn't have compiled executor!");

Is it a final solution or temp fix for the issue described in the comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking for isSliceScatter op may not be needed, instead it would make more sense to check for const inputs. for start/stop/step.
For other slice ops prepareParams is called in 2 ways:

  1. If params are const then prepareParams will be called by createPrimitive,
  2. If params are not const, createPrimitive will not be called, however since Slice/StridedSlice requires values from params to determine output shape, this node will be dynamic, thus it will call prepareParams in updateDynamicParams

In SliceScatter there is a problem with option 2 because output shape doesn't depend on data. So this would be an workaround for case where all inputs would have static shapes (node is static so no updateDynamicParams) and start/stop/step would be non-constant (prepareParams cannot be called during createPrimitive due to depending on const value).

As for final solution, I guess it may benefit from some changes to StridedSliceCommonExecutor since with static shapes, some calculations could be prepared in createPrimitive stage.

@dmitry-gorokhov
Copy link
Contributor

@maxnick Could you please take a look from CPU side?

Copy link
Contributor

@praasz praasz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM shape inference core part.

@maxnick
Copy link
Contributor

maxnick commented Dec 4, 2024

@mitruska , do you have any further comments or suggestions?

@mitruska
Copy link
Contributor

mitruska commented Dec 4, 2024

@mitruska , do you have any further comments or suggestions?

No further comments

Comment on lines 328 to 336
if (!execPtr) {
if (!isDynamicNode() && !hasConstAttrInputs) {
// SliceScatter due to not having data dependency on shape may not call prepareParams when start/stop/step values are non-constant in Static execution.
// In Slice and SliceScatter op, prepareParams would be called by createPrimitive (if const inputs) or by updateDynamicParams in case of dynamic node.
StridedSlice::prepareParams();
} else {
OPENVINO_THROW(errorPrefix, "doesn't have compiled executor!");
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the data on the corresponding inputs are not constant, it means that the executor must be reinitialized every inference call, as the data in variables may be updated each inference. If the tests pass, that means that either the test converge is still not sufficient, or there is no real data dependency and the executor may be created at the compilation stage.

@@ -83,7 +83,13 @@ class SliceScatterLayerCPUTest : public testing::WithParamInterface<SliceScatter
in_data);
} else {
// Fill the slice input2~input5 with specified data.
tensor = ov::Tensor{ov::element::i64, targetInputStaticShapes[i], inputValues[i - 2]};
auto inputValue = inputValues[i - 2];
if (!inputValue) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please elaborate on the reason behind introducing this check? It looks like the current test configuration doesn't provide null pointers. But if it does, an input tensor is created but not initialized with valid values. So what is the expectation of using such an input?

const auto param = ov::as_type_ptr<const ov::op::v0::Parameter>(funcInput.get_node_shared_ptr());
tensor = ov::Tensor{ov::element::i64, targetInputStaticShapes[i]};
} else {
tensor = ov::Tensor{ov::element::i64, targetInputStaticShapes[i], inputValue};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this is the reason why the tests don't cover the main issue, as is being resolved in this PR. The thing is that the SL tests in the static shapes configuration perform only one inference, thus the values on the variable inputs don't really change from infer to infer and SL tests don't spot the issue. But if we had had such tests, they would have revealed the design flaw of the current solution solution (please see the comment regarding execute method implementation). So apparently the SL tests should be extended to cover such a use case: the values on the variable inputs actually changes each inference. To this end even a dedicated test class may be introduced, when necessary.

Signed-off-by: Mateusz Mikolajczyk <[email protected]>
…into mateuszm/slicescatter/cpu_bugfix

Signed-off-by: Mateusz Mikolajczyk <[email protected]>
Signed-off-by: Mateusz Mikolajczyk <[email protected]>
@mmikolajcz mmikolajcz marked this pull request as draft December 18, 2024 17:12
Copy link
Contributor

github-actions bot commented Jan 2, 2025

This PR will be closed in a week because of 2 weeks of no activity.

@github-actions github-actions bot added the Stale label Jan 2, 2025
@github-actions github-actions bot removed the Stale label Jan 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: Core OpenVINO Core (aka ngraph) category: CPU OpenVINO CPU plugin category: TEMPLATE OpenVINO Template plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants