Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is there a way to run this and test? #2

Open
AmbarishGK opened this issue Oct 27, 2024 · 12 comments
Open

Is there a way to run this and test? #2

AmbarishGK opened this issue Oct 27, 2024 · 12 comments

Comments

@AmbarishGK
Copy link

I am looking into real-time object detection with Lidar pointcloud and I came across this. So is this the right repo and is there a way to test it?
@1760385452
Copy link

这个代码有人成功实现了吗，为什么依赖一直报错

@ragibarnab
Copy link
Owner

Sorry, this is a project I did back in college. I believe what you have to do is to follow the installation instructions in the README of the PointPillars repo I made this wrapper over. You have to make sure it compiles correctly and then it should be able to run the object detector. Then you need to compile it as a ROS 2 package. My code is messy and newbie-ish so take it with a grain of salt. Idk why I got so many stars lol, I was just prototyping some object detection nodes for an undergrad self-driving project.

@1760385452
Copy link

谢谢您在百忙之中的回复，我已经按照您所给链接训练好了自己的pointpillars模型，得到了自己的权重，但是我想要写一个ros2节点包来调用激光雷达进行实时检测，一直没有进展，看了您的这个ROS 2 node wrapper around the PyTorch implementation of PointPillars的源码，想要复现失败了，请问您还有这种封装pointpillars模型调用激光雷达实时检测的ros2代码吗，可以跟我交流一下吗，万分感谢！！

@ragibarnab
Copy link
Owner

Okay, so here's what I remember from back then:

The source code for the object detection node is under src/lidar_object_detection/lidar_object_detection. I put all the PointPillars code in there as a submodule for convenience. What you need to do is (after installing necessary python packages) is to compile the src/lidar_object_detection/lidar_object_detection/ops cuda code for faster voxelization necessary for running the detector. After that, you compile the ROS 2 packages using colcon (I assume you know how to do this, if not just look at ROS 2 wiki).

Under launch/main.launch.py specifies nodes that need to be launched to produce object detection inference on RViz2 from a sample of the KITTI lidar dataset.

I put 10 samples under src/lidar_object_detection/lidar_object_detection/data as dummy data. The node lidar_publisher_node defined in src/lidar_object_detection/lidar_object_detection/lidar_publisher_node.py will publish that dummy data.

The node lidar_object_detector_node defined in src/lidar_object_detection/lidar_object_detection/lidar_object_detector_node.py will take those published lidar msgs, perform inference using the PointPillars submodule, and spit out the results in a custom msg defined in src/object_detection_msgs. As for loading the weights, I put pretrained weights under src/lidar_object_detection/lidar_object_detection/weights/epoch_160.pth, and the code should automatically reference that.

For visualization, the published bounding boxes by the object detection node will be parsed by object3d_visualizer_node defined under src/object_visualization/object_visualization/object3d_visualizer_node.py. It will output markers that you can see in RViz2, showing the 3d bounding box.

I hope this helps. Remember that I created this code over a weekend in college, well over a year ago. I don't remember all the details, and the code is not the cleanest.

Ragib

@1760385452
Copy link

1760385452 commented Jan 8, 2025 via email

@ragibarnab
Copy link
Owner

Just paste the error you have and I can look at it. However, I won't be able to reproduce it on my end because I do not have a linux machine available to test the code. But I could help you debug it.

@1760385452
Copy link

1760385452 commented Jan 9, 2025 via email

@ragibarnab
Copy link
Owner

  1. I don't understand what you are asking here, maybe it got lost in translation?
  2. I'm assuming you are asking how to launch individual nodes. After building and sourcing the packages, you would run ros2 run package_name node_name, so for example to run the object detection node: ros2 run lidar_object_detection lidar_object_detector_node.
  3. Yes, the object_detection_msgs need to be compiled. You have to compile everything together with colcon so dependencies are properly found.
  4. Again, every package needs to be compiled using colcon and sourced in order for the everything to work, including ros2_numpy. I would not worry about the role of each file, just compile and run, that's the main idea.

These are very basic questions, all of which can be answered by following the wiki. You do not need tons of computer knowledge to go through and understand this material. In this "field" you have to be very self-reliant, because a lot of this material is not taught in schools, and even regular computer science students will not understand this material without going through the wiki. But I promise, it's not that hard to pick up, ROS was designed to be simple. As always, google is your best friend.

@1760385452
Copy link

1760385452 commented Jan 11, 2025 via email

@ragibarnab
Copy link
Owner

The object3d_visualizer_node will publish markers that can be seen in RViz. I don't know about Open3D.

@1760385452
Copy link

1760385452 commented Jan 15, 2025 via email

@ragibarnab
Copy link
Owner

https://robotics.stackexchange.com/questions/79779/importerror-no-module-named-em-error

Follow this link and try any solutions you can find.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@AmbarishGK @ragibarnab @1760385452