ROS1 fixes for no output for RGBD hand gesture recognition (opendr-eu#343)

* Change from TimeSynchronizer to ApproximateTimeSynchronizer

* Update rgbd-hand-gesture-learner.md

Add information about gesture class and learner output ID correspondence.

* Upload hand gesture examples

* Delete hand_gesture_example.png

* Upload hand gesture image with modified image name

* Update rgbd-hand-gesture-learner.md

Add hand gesture example image

Co-authored-by: Kostas Tsampazis <[email protected]>

Author: minhquoc0712

docs/reference/images/hand_gesture_examples.png

@@ -2,6 +2,20 @@
 
 The *rgbd_hand_gesture_learner* module contains the *RgbdHandGestureLearner* class, which inherits from the abstract class *Learner*.
 
+On the table below you can find the gesture classes and their corresponding IDs:
+
+| **ID** |    0   |   1   |   2  |   3  |        4       |        5       |   6  |  7  |   8   |   9   |  10 |  11  |   12  |    13   |  14 |   15  |
+|:------:|:------:|:-----:|:----:|:----:|:--------------:|:--------------:|:----:|:---:|:-----:|:-----:|:---:|:----:|:-----:|:-------:|:---:|:-----:|
+|  Class | COLLAB | Eight | Five | Four | Horiz HBL, HFR | Horiz HFL, HBR | Nine | One | Punch | Seven | Six | Span | Three | TimeOut | Two | XSign |
+
+The naming convention of the gestures classes is as follow:
+- V is used for vertical gestures, while H is used for horizontal gestures.
+- F identifies the version of the gesture where the front of the hand is facing the camera, while B identifies the version where the back of the hand is facing the camera.
+- R is used for right-hand gestures, while L is used for left-hand gestures.
+
+Below is an illustration image of hand gestures, the image is copied from [[1]](#dataset).
+![Hand gesture examples](images/hand_gesture_examples.png)
+
 ### Class RgbdHandGestureLearner 
 Bases: `opendr.engine.learners.Learner`
 

docs/reference/rgbd-hand-gesture-learner.md

@@ -34,7 +34,7 @@ class RgbdHandGestureNode:
 
     def __init__(self, input_rgb_image_topic="/kinect2/qhd/image_color_rect",
                  input_depth_image_topic="/kinect2/qhd/image_depth_rect",
-                 output_gestures_topic="/opendr/gestures", device="cuda"):
+                 output_gestures_topic="/opendr/gestures", device="cuda", delay=0.1):
         """
         Creates a ROS Node for gesture recognition from RGBD. Assuming that the following drivers have been installed:
         https://github.com/OpenKinect/libfreenect2 and https://github.com/code-iai/iai_kinect2.
@@ -46,10 +46,13 @@ def __init__(self, input_rgb_image_topic="/kinect2/qhd/image_color_rect",
         :type output_gestures_topic: str
         :param device: device on which we are running inference ('cpu' or 'cuda')
         :type device: str
+        :param delay: Define the delay (in seconds) with which rgb message and depth message can be synchronized
+        :type delay: float
         """
 
         self.input_rgb_image_topic = input_rgb_image_topic
         self.input_depth_image_topic = input_depth_image_topic
+        self.delay = delay
 
         self.gesture_publisher = rospy.Publisher(output_gestures_topic, Classification2D, queue_size=10)
 
@@ -75,7 +78,8 @@ def listen(self):
         image_sub = message_filters.Subscriber(self.input_rgb_image_topic, ROS_Image, queue_size=1, buff_size=10000000)
         depth_sub = message_filters.Subscriber(self.input_depth_image_topic, ROS_Image, queue_size=1, buff_size=10000000)
         # synchronize image and depth data topics
-        ts = message_filters.TimeSynchronizer([image_sub, depth_sub], 10)
+        ts = message_filters.ApproximateTimeSynchronizer([image_sub, depth_sub], queue_size=10, slop=self.delay,
+                                                         allow_headerless=True)
         ts.registerCallback(self.callback)
 
         rospy.loginfo("RGBD gesture recognition node started!")
@@ -137,6 +141,8 @@ def preprocess(self, rgb_image, depth_image):
                         type=str, default="/opendr/gestures")
     parser.add_argument("--device", help="Device to use (cpu, cuda)", type=str, default="cuda",
                         choices=["cuda", "cpu"])
+    parser.add_argument("--delay", help="The delay (in seconds) with which RGB message and"
+                        "depth message can be synchronized", type=float, default=0.1)
 
     args = parser.parse_args()
 
@@ -156,5 +162,7 @@ def preprocess(self, rgb_image, depth_image):
 
     gesture_node = RgbdHandGestureNode(input_rgb_image_topic=args.input_rgb_image_topic,
                                        input_depth_image_topic=args.input_depth_image_topic,
-                                       output_gestures_topic=args.output_gestures_topic, device=device)
+                                       output_gestures_topic=args.output_gestures_topic, device=device,
+                                       delay=args.delay)
+
     gesture_node.listen()