Align the color image frame with the depth image frame

[MQ-D-729]

Hello, is there an example update for the align color image and depth image topic `posted, similar to“/camera/aligh_depth/image" 480X270 and "/camera/align_color/image_raw" 480X270.

[felixmaisonneuve]

Hi @MQ-D-729,

You can look into the feature/calibration-program for a program and instructions that will allow you to align the depth sensor and color sensor images in moveIt.

It automates a bunch of stuff that will help you get the perfect alignement, but in the end, it just changes the parameters here :

ros_kortex_vision/launch/kinova_vision_rgbd.launch

Line 97 in fb76366

args="-0.0195 -0.005 0 0 0 0 $(arg camera_link_frame_id) $(arg depth_frame_id)" />

If you are not using MoveIt and you are trying to align the actual intel sensor, I know it is possible to disassemble the vision module to access and the USB-c port that can allow you to connect to the camera directly. You can then use the Intel software to configure you camera. I know a user who did something similar. I think it is possible to adjust the alignement from there, but I might be wrong since the depth sensor and the color sensor are not the same camera (I am not 100% sure about that part).

Let me know if this is not what you are looking for,

Best,
Felix

[Liuzhuoqiang]

may i ask where to find the feature/calibration-program for a program and instructions

[MQ-D-729]

Hi @MQ-D-729,

You can look into the feature/calibration-program for a program and instructions that will allow you to align the depth sensor and color sensor images in moveIt.

It automates a bunch of stuff that will help you get the perfect alignement, but in the end, it just changes the parameters here :

ros_kortex_vision/launch/kinova_vision_rgbd.launch

Line 97 in fb76366

args="-0.0195 -0.005 0 0 0 0 $(arg camera_link_frame_id) $(arg depth_frame_id)" />

If you are not using MoveIt and you are trying to align the actual intel sensor, I know it is possible to disassemble the vision module to access and the USB-c port that can allow you to connect to the camera directly. You can then use the Intel software to configure you camera. I know a user who did something similar. I think it is possible to adjust the alignement from there, but I might be wrong since the depth sensor and the color sensor are not the same camera (I am not 100% sure about that part).

Let me know if this is not what you are looking for,

Best, Felix

This can indeed align the color image and the depth image at a fixed position, but it only changes the TF transformation of the depth camera relative to the color camera. This transformation should be fixed, and their relative positions remain unchanged. To get an aligned image, I think It needs the camera's distortion parameters, external parameters, internal parameters, and alignment through image algorithms, similar to pyrealsenses. It seems to be possible to use usb control after disassembly, but I am worried about disassembly. I am not very good at it, so it will be more troublesome. Is there any other way?

[MQ-D-729]

may i ask where to find the feature/calibration-program for a program and instructions
inside the branch，but,use ubnutu20.04

[felixmaisonneuve]

I do not know much about camera parameters.
All the vision parameters that can be configured are detailed in the documentation.
If these options are not enough, there is nothing you could do via ROS or the Kortex API.

If you decide to proceed to disassemble the vision module to access the camera directly, we can provide the instructions. You need to ask [email protected]

Also, the ros_kortex_vision was not upgraded to ROS Noetic (ROS version of Ubuntu20.04), so you cannot currently use the repo and the calibration tool with Ubuntu20.04.
I do not know how much work this would be, but if you decide to do it, you can look at this commit where I did the Melodic -> Noetic migration on our main repo.

Hope this helps a bit,
Felix

[MQ-D-729]

The official Intel D410 depth camera shows that it supports 1280x720 at 30fps, why is there only a 480X270 depth image in the launch file. And the color camera is still OV5640, not so easy to align depth frames。

[felixmaisonneuve]

The 1280x780 option is not supported by the API for the depth sensor. This is something that we are aware of and I bug has been open for this, but I do not have a timeline for the fix