VINS-Mono-superpoint

VINS-Mono support superpoint to extract features

26 Feb 2023: first commit, support superpoint, but the real-time performance need to be improved because of too many num of features.

1. Prerequisites

1.1 Ubuntu and ROS Ubuntu 16.04. ROS Kinetic. ROS Installation additional ROS pacakge

    sudo apt-get install ros-YOUR_DISTRO-cv-bridge ros-YOUR_DISTRO-tf ros-YOUR_DISTRO-message-filters ros-YOUR_DISTRO-image-transport

1.2. Ceres Solver Follow Ceres Installation, remember to make install. (Our testing environment: Ubuntu 16.04, ROS Kinetic, OpenCV 3.3.1, Eigen 3.3.3)

1.3. onnxruntime
Build from source (tested on Ubuntu 20.04)

git clone https://github.com/microsoft/onnxruntime.git
cd onnxruntime
sh build.sh

or follow Onnxruntime Installation

1.4. superpoint.onnx A superpoint.pth file should be trained and transformed to superpoint.onnx file.
Follow Superpoint to train.
Follow onnx_runtime_cpp to transform.

2. Build VINS-Mono on ROS

Clone the repository and catkin_make:

    cd ~/catkin_ws/src
    git clone https://github.com/llyymmoo/VINS-Mono-superpoint.git
    cd ../
    catkin_make
    source ~/catkin_ws/devel/setup.bash

3. Visual-Inertial Odometry and Pose Graph Reuse on Public datasets

Download EuRoC MAV Dataset. Although it contains stereo cameras, we only use one camera. The system also works with ETH-asl cla dataset. We take EuRoC as the example.

3.1 visual-inertial odometry and loop closure

3.1.1 Open three terminals, launch the vins_estimator , rviz and play the bag file respectively. Take MH_01 for example

    roslaunch vins_estimator euroc.launch 
    roslaunch vins_estimator vins_rviz.launch
    rosbag play YOUR_PATH_TO_DATASET/MH_01_easy.bag 

(If you fail to open vins_rviz.launch, just open an empty rviz, then load the config file: file -> Open Config-> YOUR_VINS_FOLDER/config/vins_rviz_config.rviz)

3.1.2 (Optional) Visualize ground truth. We write a naive benchmark publisher to help you visualize the ground truth. It uses a naive strategy to align VINS with ground truth. Just for visualization. not for quantitative comparison on academic publications.

    roslaunch benchmark_publisher publish.launch  sequence_name:=MH_05_difficult

(Green line is VINS result, red line is ground truth).

3.1.3 (Optional) You can even run EuRoC without extrinsic parameters between camera and IMU. We will calibrate them online. Replace the first command with:

    roslaunch vins_estimator euroc_no_extrinsic_param.launch

No extrinsic parameters in that config file. Waiting a few seconds for initial calibration. Sometimes you cannot feel any difference as the calibration is done quickly.