Skip to content

Commit

Permalink
Add docs
Browse files Browse the repository at this point in the history
  • Loading branch information
jian-dong committed Apr 22, 2024
1 parent 425813f commit 4a23130
Show file tree
Hide file tree
Showing 15 changed files with 318 additions and 4 deletions.
40 changes: 40 additions & 0 deletions docs/align_depth_color.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
## Aligning Depth to Color in ROS 2

This section explains how to align depth images with color images to create an overlay image using ROS 2. This is particularly useful for applications requiring synchronized visual information from different sensor modalities.

### Commands to Align and View Depth and Color Images

1. **Basic Depth to Color Alignment:**

To simply align the depth image to the color image, use the following command:

```bash
ros2 launch orbbec_camera gemini2R.launch.py depth_registration:=true
```

This command activates the depth registration feature without opening a viewer.

2. **Viewing Depth to Color Overlay:**

If you wish to view the depth to color overlay, you need to enable the viewer by using the command below:

```bash
ros2 launch orbbec_camera gemini2R.launch.py depth_registration:=true enable_d2c_viewer:=true
```

This launches the camera node with depth to color registration and opens a viewer to display the overlay image.

### Selecting Topics in RViz2

To visualize the aligned images in RViz2:

1. Launch RViz2 after running one of the above commands.
2. Select the topic for the depth to color overlay image. An example topic selection is shown here:

![Topic Selection for Depth to Color Overlay](./images/image3.png)

### Example of Depth to Color Overlay

After selecting the appropriate topic in RViz2, you will be able to see the depth to color overlay image. Here's what it might look like:

![Depth to Color Overlay Image](./images/image4.jpg)
Binary file added docs/images/image1.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/image2.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/image3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/image4.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/image5.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/image6.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
108 changes: 108 additions & 0 deletions docs/multi_camera.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
## Using Multiple Cameras with the Orbbec ROS 2 Package

This section describes how to configure and use multiple Orbbec cameras simultaneously in a ROS 2 environment.

### Identifying Camera USB Ports

#### Script to List Connected Cameras

To determine which USB ports the cameras are connected to, you can use the following bash script. This script lists all Orbbec devices attached to the system along with their USB port and serial number.

```bash
#!/bin/bash

VID="2bc5"

for dev in /sys/bus/usb/devices/*; do
if [ -e "$dev/idVendor" ]; then
vid=$(cat "$dev/idVendor")
if [ "$vid" == "${VID}" ]; then
port=$(basename $dev)
product=$(cat "$dev/product" 2>/dev/null) # product name
serial=$(cat "$dev/serial" 2>/dev/null) # serial number
echo "Found Orbbec device $product, usb port $port, serial number $serial"
fi
fi
done
```

Save this script to a file and execute it in your terminal to output a list of connected cameras.

### Launching Multiple Cameras

#### Setup for Multiple Camera Launch

You can launch multiple cameras by specifying different USB ports for each camera. Below is an example Python script that uses the ROS 2 launch system to start two cameras with individual configurations.

```python
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription, GroupAction, ExecuteProcess
from launch.launch_description_sources from ament_index_python.packages import get_package_share_directory
import os

def generate_launch_description():
package_dir = get_package_share_directory('orbbec_camera')
launch_file_dir = os.path.join(package_dir, 'launch')

launch1_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(os.path.join(launch_file_dir, 'gemini2R.launch.py')),
launch_arguments={'camera_name': 'camera_01', 'usb_port': '2-3.4.4.4.1', 'device_num': '2', 'sync_mode': 'free_run'}.items()
)

launch2_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(os.path.join(launch_file_dir, 'gemini2R.launch.py')),
launch_arguments={'camera_name': 'camera_02', 'usb_port': '2-3.4.4.4.3', 'device_num': '2', 'sync_mode': 'free_run'}.items()
)

ld = LaunchDescription([
GroupAction([launch1_include]),
GroupAction([launch2_include])
])

return ld
```

#### Running the Launch File

To execute the launch configuration for multiple cameras, use the command:

```bash
ros2 launch orbbec_camera multi_camera.launch.py
```

### Configuring the TF Tree for Multiple Cameras

#### Example TF Configuration for Two Cameras

When using multiple cameras, it's essential to calibrate them and publish a static TF tree for each camera. The following Python script configures the TF tree based on your calibration results:

```python
from launch import LaunchDescription
from launch_ros.actions import Node

def generate_launch_description():
ld = LaunchDescription([
Node(
package='tf2_ros',
executable='static_transform_publisher',
name='camera_01_tf',
arguments=['0', '0', '0', '0', '0', '0', 'base_link', 'camera_01_link'],
output='screen'
),
Node(
package='tf2_ros',
executable='static_transform_publisher',
name='camera_02_tf',
arguments=['0', '0', '0', '0', '0', '0', 'base_link', 'camera_02_link'],
output='screen'
)
])

return ld
```

Save this configuration as `multi_camera_tf.launch.py` in the launch directory of the Orbbec camera package. To run it, use:

```bash
ros2 launch orbbec_camera multi_camera_tf.launch.py
```
53 changes: 53 additions & 0 deletions docs/point_cloud.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
## Enabling and Visualizing Point Cloud in ROS 2

This section demonstrates how to enable point cloud data output from the camera node and visualize it using RViz2, similarly to the initial camera node setup discussed in the [Starting Camera Node](./start_camera_node.MD) document.

### Enabling Depth Point Cloud

#### Command to Enable Depth Point Cloud

To activate the point cloud data stream for depth information, use the following command:

```bash
ros2 launch orbbec_camera gemini2R.launch.py enable_point_cloud:=true
```

#### Visualizing Depth Point Cloud in RViz2

After running the above command, perform the following steps to visualize the depth point cloud:

1. Open RViz2.
2. Add a `PointCloud2` display.
3. Select the `/camera/depth/points` topic for visualization.
4. Set the fixed frame to `camera_link` to properly align the data.

##### Example Visualization

Here is what the depth point cloud might look like in RViz2:

![Depth Point Cloud Visualization](./images/image5.jpg)

### Enabling Colored Point Cloud

#### Command to Enable Colored Point Cloud

To enable the colored point cloud feature, enter the following command:

```bash
ros2 launch orbbec_camera gemini2R.launch.py enable_colored_point_cloud:=true
```

#### Visualizing Colored Point Cloud in RViz2

To visualize the colored point cloud data:

1. Launch RViz2 following the command execution.
2. Add a `PointCloud2` display panel.
3. Choose the `/camera/depth_registered/points` topic from the list.
4. Ensure the fixed frame is set to `camera_link`.

##### Example Visualization

The result of the colored point cloud in RViz2 should look similar to this:

![Colored Point Cloud Visualization](./images/image6.jpg)
83 changes: 83 additions & 0 deletions docs/start_camera_node.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
## Starting the Camera Node in ROS 2

This guide provides instructions on how to launch the camera node with a colored point cloud feature enabled using ROS 2.

### Command to Start the Node

To start the camera node, execute the following command in your terminal:

```bash
ros2 launch orbbec_camera gemini2R.launch.py enable_colored_point_cloud:=true
```

This command initiates the camera node and enables the colored point cloud.

### Published Topics

Once the camera node is running, it will publish data on several ROS topics. Below is a list of the available topics:

- **IMU and Gyro Information:**
- `camera/accel/imu_info`
- `camera/gyro/imu_info`
- `camera/gyro_accel/sample`

- **Color Camera Topics:**
- `/camera/color/camera_info`
- `/camera/color/image_raw`
- `/camera/color/image_raw/compressed`
- `/camera/color/image_raw/compressedDepth`
- `/camera/color/image_raw/theora`
- `/camera/color/metadata`

- **Depth Camera Topics:**
- `/camera/depth/camera_info`
- `/camera/depth/image_raw`
- `/camera/depth/image_raw/compressed`
- `/camera/depth/image_raw/compressedDepth`
- `/camera/depth/image_raw/theora`
- `/camera/depth/metadata`
- `/camera/depth/points`
- `/camera/depth_filter_status`
- `/camera/depth_registered/points`
- `/camera/depth_to_color`
- `/camera/depth_to_left_ir`
- `/camera/depth_to_right_ir`

- **Infrared Camera Topics:**
- `/camera/left_ir/camera_info`
- `/camera/left_ir/image_raw`
- `/camera/left_ir/image_raw/compressed`
- `/camera/left_ir/image_raw/compressedDepth`
- `/camera/left_ir/image_raw/theora`
- `/camera/left_ir/metadata`
- `/camera/right_ir/camera_info`
- `/camera/right_ir/image_raw`
- `/camera/right_ir/image_raw/compressed`
- `/camera/right_ir/image_raw/compressedDepth`
- `/camera/right_ir/image_raw/theora`
- `/camera/right_ir/metadata`

- **Miscellaneous Topics:**
- `/diagnostics`
- `/parameter_events`
- `/rosout`
- `/rosout_agg`

### Visualizing Data in RViz2

To view the PointCloud or Image data, use RViz2:

1. Launch RViz2.
2. Select the topic you wish to visualize from the list of published topics.
3. Add the selected topic to RViz2 to start viewing the data.

### Example Visualizations

Here are examples of how the visualization might appear in RViz2:

- **PointCloud Visualization**
![PointCloud View](./images/image1.jpg)

- **Image Data Visualization**
![Image Data View](./images/image2.jpg)

3 changes: 2 additions & 1 deletion orbbec_camera/SDK/include/libobsensor/h/Pipeline.h
Original file line number Diff line number Diff line change
Expand Up @@ -180,9 +180,10 @@ ob_camera_param ob_pipeline_get_camera_param_with_profile(ob_pipeline *pipeline,
ob_camera_param ob_pipeline_get_camera_param(ob_pipeline *pipeline, ob_error **error);

/**
* @brief Get device calibration parameters
* @brief Get device calibration parameters with the specified configuration
*
* @param[in] pipeline pipeline object
* @param[in] config The pipeline configuration
* @param[out] error Log error messages
* @return ob_calibration_param The calibration parameters
*/
Expand Down
4 changes: 2 additions & 2 deletions orbbec_camera/SDK/include/libobsensor/h/Sensor.h
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ ob_filter_list *ob_sensor_get_recommended_filter_list(ob_sensor *sensor, ob_erro
/**
* @brief Get the number of recommended filter list
*
* @param ob_filter_list Recommended filter list
* @param filter_list Recommended filter list
* @param error Log error messages
* @return uint32_t The number of list
*/
Expand All @@ -50,7 +50,7 @@ uint32_t ob_filter_list_get_count(ob_filter_list *filter_list, ob_error **error)
/**
* @brief Get the number of recommended filter list
*
* @param ob_filter_list Recommended filter list
* @param filter_list Recommended filter list
* @param index Recommended filter index
* @param error Log error messages
* @return ob_filter The index of ob_filter
Expand Down
29 changes: 29 additions & 0 deletions orbbec_camera/SDK/include/libobsensor/hpp/Device.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -587,12 +587,41 @@ class OB_EXTENSION_API Device {
*/
void loadPresetFromJsonFile(const char *filePath);

/**
* @brief Load custom preset from data.
* @brief After loading the custom preset, the settings in the custom preset will set to the device immediately.
* @brief After loading the custom preset, the available preset list will be appended with the custom preset and named as the @ref presetName.
*
* @attention The user should ensure that the custom preset data is adapted to the device and the settings in the data are valid.
* @attention It is recommended to re-read the device settings to update the user program temporarily after successfully loading the custom preset.
*
* @param data The custom preset data.
* @param size The size of the custom preset data.
*/
void loadPresetFromJsonData(const char *presetName, const uint8_t *data, uint32_t size);

/**
* @brief Export current device settings as a preset json file.
* @brief The exported preset file can be loaded by calling @ref loadPresetFromJsonFile to restore the device setting.
* @brief After exporting the preset, a new preset named as the @ref filePath will be added to the available preset list.
*
* @param filePath The path of the preset file to be exported.
*/
void exportSettingsAsPresetJsonFile(const char *filePath);

/**
* @brief Export current device settings as a preset json data.
* @brief After exporting the preset, a new preset named as the @ref presetName will be added to the available preset list.
*
* @attention The memory of the data is allocated by the SDK, and will automatically be released by the SDK.
* @attention The memory of the data will be reused by the SDK on the next call, so the user should copy the data to a new buffer if it needs to be
* preserved.
*
* @param[out] data return the preset json data.
* @param[out] dataSize return the size of the preset json data.
*/
void exportSettingsAsPresetJsonData(const char *presetName, const uint8_t **data, uint32_t *dataSize);

friend class Pipeline;
friend class Recorder;
friend class CoordinateTransformHelper;
Expand Down
2 changes: 1 addition & 1 deletion orbbec_camera/SDK/include/libobsensor/hpp/Filter.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -486,7 +486,7 @@ template <typename T> bool Filter::is() {
if(name == "ThresholdFilter") {
return typeid(T) == typeid(ThresholdFilter);
}
if(name == "Disparity to Depth") {
if(name == "DisparityTransform") {
return typeid(T) == typeid(DisparityTransform);
}
if(name == "NoiseRemovalFilter") {
Expand Down
Binary file modified orbbec_camera/SDK/lib/arm64/libOrbbecSDK.so.1.9.2
Binary file not shown.

0 comments on commit 4a23130

Please sign in to comment.