Comparing LiDAR Based SLAM Trajectory Error with GPS/INS Trajectory Error

[yvzksgl]

In the mapping tests, trajectory errors of LiDAR based SLAM algorithms cannot be obtained. In order to calculate trajectory error of SLAM algorithms, ground truth should be generated and GPS/INS can be used for this purpose.

Possible solution:

• Determine SLAM packages that work without the need for any additional sensor from LiDAR.
• Obtain trajectories from SLAM algorithms.
• Generate the ground truth according to the trajectory obtained from GPS/INS and its error.
• Calculate the trajectory error of the SLAM algorithm and compare trajectory errors with GPS/INS’s trajectory error in terms of position(x, y, z) and orientation(roll, pitch, yaw).
• Test with IMU-LiDAR SLAM algorithms using the GPS&INS IMU data.

It will provide numerical metrics for the accuracy of SLAM algorithms.

[n-patiphon]

Hi, a member of the Mapping WG just evaluated various open-source SLAM algorithms (many of which require only LiDAR) and published the results here.

The document might be related to this discussion. 😀

[yvzksgl]

Hi, thanks for your contrubition i was already checking there for selecting SLAM algorithm. 😀

[n-patiphon]

Also, about the third point on calcuating the trajectory error, MAP IV has open-sourced a simple Python script to calculate and plot error between two or more trajectories (stored in CSV files or ROSBAGs).

This is the repo, in case you are interested.

[yvzksgl]

In my case ground truth is stored in .csv and .txt files and the trajectory to be compared is a rosbag record. Converting rosbag to .csv file should not be challenging but I am trying to keep it integrated with ROS. Also I was planning to calculate and publish the trajectory error of the SLAM algorithm instantly and publish the overall error at the end of the rosbag record. I would be greatful if there is a suggestion. Thank you.

[n-patiphon]

As you said, converting a ROSBAG file to a CSV file would be one way to do it.

Alternatively, you could convert the ground truth from CSV to ROSBAG (either geometry_msgs/msg/PoseWithCovarianceStamped or nav_msgs/msg/Odometry) instead. Since localization_evaluation_toolkit supports comparing trajectories stored in ROSBAGs, you can directly use it that way.

You could also create an issue in the repo regarding this functionality. It might be added in the future.