[Bug]: preprocess set_spatial_static_shape compile fail on NPU

[personball]

OpenVINO Version

2024.5

Operating System

Windows System

Device used for inference

NPU

Framework

None

Model used

palm_detection_192_lite

Issue description

		// palm model
		auto palmOvModel = core.read_model(config->palmDetectorXmlPath, config->palmDetectorBinPath);

		if (pd_input_size == 128) {
			pd_input_name = "input"; //128x128
		}
		else {
			pd_input_name = "input_1";//192x192
		}

		//palm_ppp
		auto palm_ppp = ov::preprocess::PrePostProcessor::PrePostProcessor(palmOvModel);
		palm_ppp.input(pd_input_name).tensor()
			.set_element_type(ov::element::u8)				    //cv::Mat default
			.set_layout(ov::Layout("NHWC"))				    //cv::Mat default 640,640,3   
			.set_spatial_static_shape(640, 640)	                            //compile fail on NPU
			.set_color_format(ov::preprocess::ColorFormat::BGR);//cv::Mat default
		palm_ppp.input(pd_input_name).model()
			.set_layout(ov::Layout("NHWC"));//palm detection 192 NHWC
		palm_ppp.input(pd_input_name).preprocess()
			.resize(ov::preprocess::ResizeAlgorithm::RESIZE_LINEAR, pd_input_size, pd_input_size)
			.convert_element_type(ov::element::f32)
			.scale({ 255.0, 255.0 , 255.0 })
			.convert_color(ov::preprocess::ColorFormat::RGB);

		cout << "palm_ppp:" << endl;
		cout << palm_ppp << endl;

		palm_model = core.compile_model(
			palm_ppp.build(),
			config->palmDetectorOVDeviceName,
			ov::hint::performance_mode(ov::hint::PerformanceMode::CUMULATIVE_THROUGHPUT),
			ov::hint::num_requests(16));
		Log("palm model loaded.");

The step .set_spatial_static_shape(640, 640) with .resize(ov::preprocess::ResizeAlgorithm::RESIZE_LINEAR, pd_input_size, pd_input_size) compile fail on NPU ( on GPU is ok)

Step-by-step reproduction

model file
palm_detection_192_lite.zip

Relevant log output

palm_ppp:
Input "input_1" (color BGR):
    User's input tensor: [1,640,640,3], [N,H,W,C], u8
    Model's expected tensor: [1,192,192,3], [N,H,W,C], f32
    Pre-processing steps (4):
      resize to (192, 192): ([1,640,640,3], [N,H,W,C], u8, BGR) -> ([1,192,192,3], [N,H,W,C], u8, BGR)
      convert type (f32): ([1,192,192,3], [N,H,W,C], u8, BGR) -> ([1,192,192,3], [N,H,W,C], f32, BGR)
      scale (255,255,255): ([1,192,192,3], [N,H,W,C], f32, BGR) -> ([1,192,192,3], [N,H,W,C], f32, BGR)
      convert color (RGB): ([1,192,192,3], [N,H,W,C], f32, BGR) -> ([1,192,192,3], [N,H,W,C], f32, RGB)

error: SetupLocationVerifier Pass failed : Found 1 duplicated names after full verification
Exception from src/inference/src/cpp/core.cpp:107:
Exception from src/inference/src/dev/plugin.cpp:54:
Exception from src/plugins/intel_npu/src/plugin/src/plugin.cpp:622:
Exception from src/plugins/intel_npu/src/plugin/src/compiled_model.cpp:61:
Check 'result == ZE_RESULT_SUCCESS' failed at src/plugins/intel_npu/src/compiler/src/zero_compiler_in_driver.cpp:746:
Failed to compile network. L0 createGraph result: ZE_RESULT_ERROR_INVALID_ARGUMENT, code 0x78000004. Compilation failed
Failed to create executable

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[avitial]

@personball it seems in your attachment only the model's .xml file was included, can you provide also the weights file (.bin file) of your model? If possible to share a sample reproducer or your application so we can test on our side. What kind of model topology is your model and what is the original framework?

It would be great also if you can check on the latest 2024.6 release and see if the issue still present.

[personball]

@personball it seems in your attachment only the model's .xml file was included, can you provide also the weights file (.bin file) of your model? If possible to share a sample reproducer or your application so we can test on our side. What kind of model topology is your model and what is the original framework?

It would be great also if you can check on the latest 2024.6 release and see if the issue still present.

palm_detection_192_lite.zip

it's google mediapipe palm model converted by ovc

[avitial]

@personball thanks for the updated model files. I've just tried this on OpenVINO 2024.6 release and the NPU Driver version 1.10.1 on Linux and its working great. Could you please try the latest driver available here for your Windows system and also upgrade to OV 2024.6? Please have a try and let us know if the issue persists. If issue continues, then I believe there might be an issue with the preprocessing step used in the application.

$ benchmark_app -m palm_detection_192_lite.xml -d NPU -t 5
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2024.6.0-17404-4c0f47d2335-releases/2024/6
[ INFO ]
[ INFO ] Device info:
[ INFO ] NPU
[ INFO ] Build ................................. 2024.6.0-17404-4c0f47d2335-releases/2024/6
[..]
[ INFO ] Model inputs:
[ INFO ]     input_1 (node: input_1) : f32 / [...] / [1,192,192,3]
[ INFO ] Model outputs:
[ INFO ]     Identity (node: 231) : f32 / [...] / [1,2016,18]
[ INFO ]     Identity_1 (node: 226) : f32 / [...] / [1,2016,1]
[Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1
[Step 6/11] Configuring input of the model
[ INFO ] Model inputs:
[ INFO ]     input_1 (node: input_1) : u8 / [N,H,W,C] / [1,192,192,3]
[ INFO ] Model outputs:
[ INFO ]     Identity (node: 231) : f32 / [...] / [1,2016,18]
[ INFO ]     Identity_1 (node: 226) : f32 / [...] / [1,2016,1]
[Step 7/11] Loading the model to the device
[ INFO ] Compile model took 563.47 ms
[Step 8/11] Querying optimal runtime parameters
[ INFO ] Model:
[ INFO ]   DEVICE_ID:
[ INFO ]   ENABLE_CPU_PINNING: False
[ INFO ]   EXECUTION_DEVICES: NPU
[ INFO ]   EXECUTION_MODE_HINT: ExecutionMode.PERFORMANCE
[ INFO ]   INFERENCE_PRECISION_HINT: <Type: 'float16'>
[ INFO ]   LOADED_FROM_CACHE: False
[ INFO ]   MODEL_PRIORITY: Priority.MEDIUM
[ INFO ]   NETWORK_NAME: TensorFlow_Lite_Frontend_IR
[ INFO ]   NPU_COMPILATION_MODE_PARAMS:
[ INFO ]   OPTIMAL_NUMBER_OF_INFER_REQUESTS: 4
[ INFO ]   PERFORMANCE_HINT: PerformanceMode.THROUGHPUT
[ INFO ]   PERFORMANCE_HINT_NUM_REQUESTS: 1
[ INFO ]   PERF_COUNT: False
[...]
[ INFO ] Execution Devices:NPU
[ INFO ] Count:            2876 iterations
[ INFO ] Duration:         5009.88 ms
[ INFO ] Latency:
[ INFO ]    Median:        6.83 ms
[ INFO ]    Average:       6.91 ms
[ INFO ]    Min:           6.46 ms
[ INFO ]    Max:           22.65 ms
[ INFO ] Throughput:   574.07 FPS

[personball]

@avitial with NPU driver 32.0.100.3104 and openvino 2024.6, we get log:

palm_ppp:
Input "input_1" (color BGR):
    User's input tensor: [1,640,640,3], [N,H,W,C], u8
    Model's expected tensor: [1,192,192,3], [N,H,W,C], f32
    Pre-processing steps (4):
      resize to (192, 192): ([1,640,640,3], [N,H,W,C], u8, BGR) -> ([1,192,192,3], [N,H,W,C], u8, BGR)
      convert type (f32): ([1,192,192,3], [N,H,W,C], u8, BGR) -> ([1,192,192,3], [N,H,W,C], f32, BGR)
      scale (255,255,255): ([1,192,192,3], [N,H,W,C], f32, BGR) -> ([1,192,192,3], [N,H,W,C], f32, BGR)
      convert color (RGB): ([1,192,192,3], [N,H,W,C], f32, BGR) -> ([1,192,192,3], [N,H,W,C], f32, RGB)

[ERROR] 14:22:57.521 [vpux-compiler] Got Diagnostic at loc(fused<{name = "module", type = "Module"}>["module"]) : SetupLocationVerifier Pass failed : Found 1 duplicated names after full verification
loc(fused<{name = "module", type = "Module"}>["module"]): error: SetupLocationVerifier Pass failed : Found 1 duplicated names after full verification
[ERROR] 14:22:57.522 [vpux-compiler] Failed Pass SetupLocationVerifier on Operation loc(fused<{name = "module", type = "Module"}>["module"])
Exception from src\inference\src\cpp\core.cpp:107:
Exception from src\inference\src\dev\plugin.cpp:53:
Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:717:
Exception from src\plugins\intel_npu\src\compiler_adapter\src\ze_graph_ext_wrappers.cpp:389:
L0 pfnCreate2 result: ZE_RESULT_ERROR_INVALID_ARGUMENT, code 0x78000004 - generic error code for invalid arguments . Compilation failed