EfficientNetB3 not working in Debug for some PCs

[cesarpgouveia]

The EfficientNetB3 model, running with OpenCV and OpenVINO frameworks, crashes during inference when the code is compiled in Debug mode. This issue is reproducible only on certain PCs and does not occur when the code is compiled in Release mode. The crash specifically happens when using the CPU plugin for inference.

Steps to Reproduce:

Use the provided example code to set up and run inference with the EfficientNetB3 model.

Compile the code in Debug mode.

Execute the program.

Expected Behavior:
The program should successfully complete the inference process without any crashes, as it does in Release mode.

Actual Behavior:
The program crashes during the net.forward() call when attempting inference in Debug mode.

Environment:

OpenCV version: 4.9.1

OpenVINO version: OpenVINO 2023.3

Operating System: Windows 10

Model: EfficientNetB3 (attached as EfficientNetB3_from_scratch.onnx)

CPU that crashes in Debug: 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz

#include <iostream>
#include <chrono>
#include <fstream>

#include <opencv2/highgui.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/dnn.hpp>
#include <opencv2/core.hpp>


std::string imageFilename = "C:\\development\\ColorImage.png";
std::string modelFilename = "C:\\development\\EfficientNetB3_from_scratch.onnx";
cv::dnn::Backend targetBackend = cv::dnn::DNN_BACKEND_INFERENCE_ENGINE;
cv::dnn::Target targetDevice = cv::dnn::DNN_TARGET_CPU;


unsigned int numInferences = 100;
cv::Size modelInputSize = cv::Size(300, 300);
cv::ImreadModes imReadMode = cv::IMREAD_COLOR;
std::vector<std::string> inputLayerNames = {"data"};
bool swapRBChannels = false;
unsigned int numChannels = 3;

int main()
{
    cv::dnn::Net net = cv::dnn::readNetFromONNX(modelFilename);

    net.setPreferableBackend(targetBackend);
    net.setPreferableTarget(targetDevice);

    cv::Mat img = cv::imread(imageFilename, imReadMode);

    cv::Mat imgResized;
    cv::resize(img, imgResized, modelInputSize);

    std::vector<cv::Mat> imgBatch = { imgResized };

    cv::Mat blob = cv::dnn::blobFromImages(imgBatch, 1.0, cv::Size(), cv::Scalar(), swapRBChannels, false, CV_32F);

    std::cout << "Blob size: " << blob.size[0] << "x" << blob.size[1] << "x" << blob.size[2] << "x" << blob.size[3] << std::endl;

    for (auto inputLayerName : inputLayerNames)
        net.setInput(blob, inputLayerName);

    std::vector<cv::String> unconnectedOutLayerNames = net.getUnconnectedOutLayersNames();

    std::vector<cv::Mat> outputs;
    outputs.clear();

    std::chrono::high_resolution_clock::time_point timeLoadModelPlusInference1 = std::chrono::high_resolution_clock::now();

    net.forward(outputs, unconnectedOutLayerNames);

    std::chrono::high_resolution_clock::time_point timeLoadModelPlusInference2 = std::chrono::high_resolution_clock::now();

    std::chrono::duration<double, std::milli> ms_doubleTimeLoadModelPlusInference = timeLoadModelPlusInference2 - timeLoadModelPlusInference1;

    std::cout << "Execution time (load model + inference): " << ms_doubleTimeLoadModelPlusInference.count() << std::endl; // in ms

    std::chrono::high_resolution_clock::time_point time1 = std::chrono::high_resolution_clock::now();

    try
    {
        for (size_t i = 0; i < numInferences; i++)
            net.forward(outputs, unconnectedOutLayerNames);
    }
    catch (std::exception& ex)
    {
        std::cout << ex.what() << std::endl;
    }
    
    std::chrono::high_resolution_clock::time_point time2 = std::chrono::high_resolution_clock::now();

    std::chrono::duration<double, std::milli> ms_double = time2 - time1;

    std::cout << "Execution time inference only: " << ms_double.count() / numInferences << std::endl; // in ms
}

[cesarpgouveia]

Model was too big, added a link for wetransfer here: https://we.tl/t-eLI8VYZajj

[wenjiew]

Hi, thanks for raising this issue. We will take a look. Just to double-check, have you tried the latest release of OpenVINO (2024.5 or 2024.6) to see whether this is fixed?

[cesarpgouveia]

Hey @wenjiew thanks for the reply. I didn't check on recent versions because I need the model to work on that specific version of the OpenVINO. Do you need me to provide the model again (the link is probably expired)?

Thanks again,
César.

[yuxu42]

Hi @chenhu-wang Could you please take a look? Thanks!

[chenhu-wang]

@cesarpgouveia The model link is not valid, could you please provide the model again. I would also confirm that you build the demo code with debug mode with OpenVINO 2023.3 installed form Release package. You didn't build OpenVINO 2023.3 by yourself in debug model, is it right? Thanks!

[cesarpgouveia]

New link: https://we.tl/t-0DSFF3Z2ep (available for the next 7 days)

No I didn't built OpenVINO 2023.3 by myself, the way I build it is by compiling OpenCV with support for OpenVINO. I simply download the desired version of OpenVINO from your pre-built repositories: https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/windows. This includes both the release and debug DLLs, which OpenCV automatically detects and uses. The exact version of my OpenVINO DLLs is: w_openvino_toolkit_windows_2023.3.0.13775.ceeafaf64f3_x86_64.

Thanks,
César.