How will the IMU integration into openvslam be implemented?

[tanjunyao7]

hi guys,
I notice that the next task would be integrating imu into the vslam system. Since Christian Forster's on-manifold preintegration strategy is referred, I guess the key estimation framework will be optimization based instead of filtering. However there remain several questions:

1. Are you going to initialize the imu sensor so that the initial states(gravity vector, scale velocities and biases) are available at the beginning? Or do you not explicitly estimate the initial states and instead dynamically correct them in the subsequent slam?
2. Is the visual inertial system tightly coupled or loosely coupled? If tightly, the imu factors and visual factors are optimized w.r.t a united cost function jointly. If loosely, the imu motion model supplies pose prediction for the frontend camera tracking, and the mapping BA result corrects for imu errors, iteratively.
3. Are imu measurements to be taken into account when loop closure occurs?

thanks for clarifying.

[ymd-stella]

the next task would be integrating imu into the vslam system.

No. The objective is to improve accuracy, robustness, functionality, etc., not to use IMUs. The use of IMU is a means to an end, not an end in itself. If there is another appropriate means, it will be implemented first.

Or do you not explicitly estimate the initial states and instead dynamically correct them in the subsequent slam?

Is that possible?

Is the visual inertial system tightly coupled or loosely coupled?

Both.

Are imu measurements to be taken into account when loop closure occurs?

It will be taken into account. I don't understand why you are having the question.

[tanjunyao7]

because I’m going to implement it and submit a PR. I need to know the expected behavior of the vi slam system.

[ymd-stella]

If you are going to work on IMU integration, please be clear about your objectives. If the objective is clear, the means will naturally be determined.

[ymd-stella]

See #228. If your objectives are matched, I recommend that you work with @stevemartinov.

[tanjunyao7]

My objective is not specific, but general. As you said, increase the system’s accuracy, robustness and reliability etc. Different algorithm variants have their own priorities and drawbacks. I guess I need to implement all the aforementioned modules and it will be up to the user to decide whether or not to enable some specific functionality.

[ymd-stella]

Continue the discussion at #235.

[tanjunyao7]

Hi bro @ymd-stella , I'm about to finish the integration. During the development, I found the original implementation has some inefficiencies, some examples of which are:

1. too many shared_ptr instances, especially config objects, across threads and data, which will undoubtedly harm the performance because the shared_ptr has to lock and unlock to update the count everytime it was copied and destructed.
2. unnecessary member copy from frame to keyframe. why not just assign a pointer of frame for every keyframe?
3. redundant memory is not released. If a duplicated keyframe is treated as removed, the object itself is not removed, but it's flag is set so that subsequent processes don't take it into account. I think this increase the complexity.
   Am I free to optimize these inefficiencies?

[ymd-stella]

too many shared_ptr instances, especially config objects, across threads and data, which will undoubtedly harm the performance because the shared_ptr has to lock and unlock to update the count everytime it was copied and destructed.

I think the impact on performance is small. Can you show any evidence that it is affecting the performance?

unnecessary member copy from frame to keyframe. why not just assign a pointer of frame for every keyframe?

I am already planning.

redundant memory is not released. If a duplicated keyframe is treated as removed, the object itself is not removed, but it's flag is set so that subsequent processes don't take it into account. I think this increase the complexity.

It is freed by local_map_cleaner.

--

No other tasks will be performed until #239 is completed, and no PullRequest will be reviewed. It would be helpful if you could reduce the number of other discussions.

[tanjunyao7]

too many shared_ptr instances, especially config objects, across threads and data, which will undoubtedly harm the performance because the shared_ptr has to lock and unlock to update the count everytime it was copied and destructed.

I think the impact on performance is small. Can you show any evidence that it is affecting the performance?

It's big for embedded systems.

[cyberbeat]

Some update for this? #239 seems closed now. @tanjunyao7 did you succeed?