High CPU usage when running yarpdev with realsense2 device

[xEnVrE]

Describe the bug
When running a yarpdev in order to get rgb/depth images from a RealSense camera, the CPU usage is quite high.

See attached screenshots.

top showing > 200% CPU usage

htop showing multiple yarpdev processes/threads spawned

---

In order to make a comparison, consider the CPU usage when running the realsense-viewer.

top showing < 100% CPU usage

htop showing multiple realsense-viewer processes/threads spawned

To Reproduce
I cloned the latest librealsense repository and compiled using these options set to ON

BUILD_EASYLOGGINGPP:BOOL=ON
BUILD_EXAMPLES:BOOL=ON
BUILD_GLSL_EXTENSIONS:BOOL=ON
BUILD_GRAPHICAL_EXAMPLES:BOOL=ON
BUILD_SHARED_LIBS:BOOL=ON
BUILD_WITH_STATIC_CRT:BOOL=ON
BUILD_WITH_TM2:BOOL=ON
CMAKE_COLOR_MAKEFILE:BOOL=ON
CMAKE_EXPORT_COMPILE_COMMANDS:BOOL=ON
ENABLE_CCACHE:BOOL=ON
FORCE_RSUSB_BACKEND:BOOL=ON
GLFW_INSTALL:BOOL=ON
HWM_OVER_XU:BOOL=ON
IMPORT_DEPTH_CAM_FW:BOOL=ON

Please note that BUILD_WITH_OPENMP was set to OFF.

Then I compiled yarp enabling realsense.

Additional context

Initially I though that the issue might be from the usage of OPENMP within the librealsense. In fact they are using it to speed up the rgb/depth alignment process. However, I checked carefully and when enabling OPENMP in the build the CPU usage increases even further to 960 %.

In order to have a fair comparison, I disabled the usage of librealsense alignment routine inside the yarp realsense driver since the realsense-viewer is not using it (I checked the source code). This, again, because I though that the issue might come from the rgb/depth alignment process.

As a third test, I also tried enabling the support to CUDA because it enables alignment computation within the GPU. However, even in the configuration CUDA=ON, OPENMP=OFF the consumption is still higher than the one of the realsense-viewer and not different from the configuration CUDA=OFF, OPENMP=OFF.

Configuration:

• OS: Ubuntu 18.04
• yarp version: c88b9f8d3ee3ef8cd9c0bc3cc425baa2cca43bcf
• compiler: g++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
• librealsense: 86280d3643c448c73c45a5393df4e2a3ddbb0d39

[Nicogene]

Thanks @xEnVrE for reporting it, I expect that the yarp device driver has an over-head respect to realasense-viewer for the presence of the streaming ports etc but it is actually too much.

I will conduct an analysis using hotspot in order to see where is the bottleneck

[xEnVrE]

It might be that the high CPU usage comes from the usage of the realsense RSUSB backend as per this issue.

cc @Nicogene

[xEnVrE]

The reason why I compiled using FORCE_RSUSB_BACKEND:BOOL=ON was that the pre-compiled package was not working properly on updated Ubuntu 18.04 installations (as we have in our lab) due to missing patches from intel side for latest kernels (>= 5.0).

However 11 days ago, DKMS packages supporting latest kernels (up to 5.3) were released (see IntelRealSense/librealsense#4586 (comment)).

Then, even if libuvc has been deprecated, we should try it because it will probably lower down the CPU usage.

cc @Nicogene

[Nicogene]

Actually we experienced issues with libuvc with a version of kernel < 5.0, but I am happy to hear that they fixed the DKMS

[xEnVrE]

However 11 days ago, DKMS packages supporting latest kernels (up to 5.3) were released (see IntelRealSense/librealsense#4586 (comment)).

Tried today very briefly as @lnobile had to install librealsense on his laptop for other reasons. The apt installation now goes smooth (no errors as in the past) but, at least on realsense-viewer, the stream of images freezes almost continuously (that is why I had to use the rsusb backend in the past). However, the CPU usage was below 50%.

[xEnVrE]

The issues with the rsusb backend have been solved in the new librealsense 2.34.0 release as per IntelRealSense/librealsense#5310.

[Nicogene]

Great news! @xEnVrE can you try to profile now the performance of our device with the latest librealsense 2.34?

[xEnVrE]

Unfortunately, I cannot right now. I will do, as soon as I am able to get a RealSense camera from the lab (which is going to happen sooner or later).

[xEnVrE]

Hi @Nicogene,

I tested with the latest release librealsense 2.35 compiled with FORCE_RSUSB_BACKEND=ON:

It seems better than before.

[PeterBowman]

I have just noticed that the realsense2_camera ROS node seems far less resource consuming than YARP's realsense2 driver.

[xEnVrE]

Hi @PeterBowman, just to understand, which version of the librealsense2 library are you building against? Thank you

[PeterBowman]

Hi, @xEnVrE. I installed the ros-melodic-realsense2-camara package which, at time of writing, consumes librealsense2.so.2.37.0 from the ROS distribution.

I was compiling YARP against the latest Intel distribution on Ubuntu Bionic (librealsense2.so.2.39.0). However, using 2.37.0 (same as ROS) has no impact on the performance of the YARP device.

[drdanz]

I'm moving this issue to the new repository where the realsense driver will be developed