hri_body_detect

Overview

⚠️ some of the links are yet to be updated and may be pointing to the original ROS page. As soon as all the involved components will be officially documented for ROS 2 as well, we will update this document.

hri_body_detect is a ROS4HRI-compatible 2D and 3D body pose estimation node.

It is built on top of Google Mediapipe 3D body pose estimation.

The node provides the 2D and 3D pose estimation for the detected humans in the scene, implementing a robust solution to self-occlusions.

This node performs the body-pose detection pipeline, publishing information under the ROS4HRI naming convention regarding the body ids (on the /humans/bodies/tracked topic), the bodies bounding box, and the jointstate of the bodys' skeleton.

To estimate the body position, the node does not need a RGB-D camera, only RGB is required. However, using RGB-D camera provides a more accurate depth estimation.

Important: to estimate the body depth without using a depth sensor, a calibrated RGB camera is required. You can follow this tutorial to properly calibrate your camera.

Launch

ros2 launch hri_body_detect hri_body_detect.launch.py <parameters>

ROS API

Parameters

Node parameters:

• image_compressed (default, False): selects the compressed image transport
• use_depth (default: False): whether or not to rely on depth images for estimating body movement in the scene. When this is False, the node estimates the body position in the scene solving a P6P problem for the face and approximating the position from this, using pinhole camera model geometry.
• stickman_debug (default: False): whether or not to publish frames representing the body skeleton directly using the raw results from mediapipe 3D body pose estimation. These debug frames are not oriented to align with the body links (ie, only the 3D location of the frame is useful).
• detection_conf_thresh (default: 0.5): threshold to apply to the mediapipe pose detection. Higher thresholds will lead to less detected bodies, but also less false positives.
• use_cmc (default: False): whether or not to enable camera motion compensation in the tracker. It compensates the movement of the camera respect to the world during tracking, but it is CPU intensive as it is computing the optical flow.

Topics

hri_body_detect follows the ROS4HRI conventions (REP-155). In particular, refer to the REP to know the list and position of the 2D/3D skeleton points published by the node.

Subscribed topics

• camera_info (sensor_msgs/CameraInfo): rgb camera meta information
• image (sensor_msgs/Image): only if image_compressed is false; rgb image, processed for body detection and 3D body pose estimation.
• image/compressed (sensor_msgs/CompressedImage): only if image_compressed is true; rgb image, processed for body detection and 3D body pose estimation; note that the suffix /compressed is added after the remapping is resolved, so you should remap only image regardless of the image_compressed value.
• depth_info (sensor_msgs/CameraInfo): depth camera meta information
• depth_image/compressed (sensor_msgs/CompressedImage): only if image_compressed is true; depth image used to estimate the 3D body position with respect to the camera; note that the suffix /compressed is added after the remapping is resolved, so you should remap only depth_image regardless of the image_compressed value.

Published topics

• /humans/bodies/tracked (hri_msgs/IdsList): list of the bodies currently detected. There will be only one body in the list.
• /humans/bodies/<body_id>/skeleton2d (hri_msgs/Skeleton2D): detected 2D skeleton points.
• /humans/bodies/<body_id>/joint_states (sensor_msgs/JointState): skeleton joints state.
• /humans/bodies/<body_id>/position: (geometry_msgs/PointStamped): filtered body position, representing the point between the hips of the tracked body. Only published when use_depth = True.
• /humans/bodies/<body_id>/velocity: (geometry_msgs/TwistStamped): filtered body velocity. Only published when use_depth = True.
• /humans/bodies/<body_id>/roi (hri_msgs/NormalizedRegionOfInterest2D): body bounding box in normalized image coordinates.
• /humans/bodies/<body_id>/urdf (std_msgs/String): body URDF.

Visualization

It is possible to visualize the results of the body pose estimation in rviz using the hri_rviz Skeleton plugin. A visualization example is reported in the image above.