Use new API calls for OpenVino

[tuxbotix]

Why? What?

Use the changed API for creating a tensor. How-To taken from their commit: intel/openvino-rs@b6eacef

Fixes #1083

ToDo / Known Issues

Ideas for Next Iterations (Not This PR)

• Add a 'smoke test' for the Object Detector to catch OpenVino setup, etc issues (Hopefully, the CI will also catch these?)

How to Test

./pepsi run should build for webots and launch properly.

[oleflb]

unfortunately this requires cloning..., opened some issues over at openvino

• 133
• 134

[tuxbotix]

I think the reason to yank is the 0.7.1, 0.7.0 releases have memory leaks and segfaults (caused by this and some other cases). Most of the 0.7.2 release is on fixing memory leaks and other memory issues -> https://github.com/intel/openvino-rs/releases/tag/v0.7.2 . While I'm not a fan of yanking a release, I can't blame them for doing this as the listed problems are quite serious.

The cost of copying is also discussed in the change of API (second link in my issue). IMHO a slightly less performant (copy is with memcpy, and aligned memory) library is better than a library that might segfault. Based on their discussions, they'll push a fix at a later time, but we don't know when will that happen.

Additionally, we can change the cycler's architecture to keep the tensor as part of the state and load the input image directly into it instead of the scratchpad.

[oleflb]

No, not possible since the Tensor does not impl Send :(

[tuxbotix]

Ouch! 😬 That's bad 😞.
I'll try to setup a benchmark today if I can to see the impact of the copy and some alternative we could try.

Based on the notes by the authors, they would need to touch a lot of code to get the lifetime information right to create a tensor from a pointer and ensure safety.

[oleflb]

For now we can just remove the scratchpad and create the Tensor in the cycle function. you can use get_data_mut to create a mutable f32 slice that you can pass to the load_into_scratchpad function.

[oleflb]

Yes, feel free to remove the trait. I'd say we merge this state with the allocation.

[oleflb]

thank you for fixing :)

[tuxbotix]

And thank you for integrating all of this and catching the things I missed!!