3D LiDAR Semantic Segmentation for Autoware.Universe

[tzu-lin-huang-128]

As mentioned in the discord channel, I am currently working on two tasks:

1. Centerpoint Object Detection with Synthetic Dataset (mainly about heavy rain, noise data, occlusion, and low-resolution data perception. Hopefully, I will have some public video clips soon)

Why low-resolution data?
Because the network can be applied to Radar data. (Radar input (low-resolution) with LiDAR output (high-resolution)

2. Semantic Segmentation (Dec. 2022) gonna integrated into Autoware.Universe. (As for Panoptic Segmentation still working in progress probably can commit to Autoware.Universe next year.). The idea is to provide semantic segmentation labels for LiDAR data so that developers can use them for sensor fusion or any other advanced usages. (Example application information will be documented in the same repo)

Note: The released code was developed in June 2022 which is a new network architecture. In 2020, I have no idea how to work with Synthetic datasets and not much practical experience with domain adoption.

Sorry, I cannot upload a new video clip for the 2022 version since all the recorded datasets and trained models cannot recover from the local server (It burned and was damaged last month due to a power issue). Gladly, the latest code and the synthetic dataset tool developed myself are intact

New Feature Lists and Information will be released here soon ...

[yukkysaito]

Centerpoint Object Detection with Synthetic Dataset (mainly about heavy rain, noise data, occlusion, and low-resolution data perception. Hopefully, I will have some public video clips soon)

I'm looking forward to it very much!

Semantic Segmentation (Dec. 2022) gonna integrated into Autoware.Universe. (As for Panoptic Segmentation still working in progress probably can commit to Autoware.Universe next year.). The idea is to provide semantic segmentation labels for LiDAR data so that developers can use them for sensor fusion or any other advanced usages. (Example application information will be documented in the same repo)

Very nice 👍
There is no semantic segmentation output type as the output of perception component because planning component difficult to use semantic segmentation output type.
Therefore, if we use semantic segmentation or panoptic segmentation output, it would help to improve 3D object recognition(dynamic object) or obstacle segmentation(obstacle pointcloud).
Example :

• Use as ground remover
• Use as a clustering of dynamic object such as pedestrian, car, bike and etc.
  We can support to integrate it.