Drift when dynamic object passes

[stevemartinov]

Hi,

I have faced a problem when mapping. The big object (sometimes even people) do pass in front of the camera while the camera is stationary and features are being extracted from those objects. Then, the camera drifts a lot. What would be a good strategy here? I do not think that IMU integration would help because after checking your IMU branch, it still takes a significant information from the extracted keypoints and IMU is not being fed into a filter.

[ymd-stella]

IMU integration is not yet complete. There are no plans to proceed with the development of IMU integration at this time.

I recommend using a camera with a wide field of view (fisheye camera) or a 360 degree panoramic camera.

[stevemartinov]

I did test the IMU branch and it does seem to work especially with a scale issue. What is not complete that really stops it from using this feature?

[ymd-stella]

Everything is unimplemented except for the data definition and preprocessing. For example, optimization.

[stevemartinov]

Ok, I will start implementing. Is there any specific paper I can follow to integrate the IMU?

[ymd-stella]

See On-Manifold Preintegration for Real-Time Visual-Inertial Odometry and Inertial-Only Optimization for Visual-Inertial Initialization.

There is an experimental implementation, but it has some issues.
https://github.com/ymd-stella/openvslam-community/tree/wip/gba.

• The edge implementation is untested and suspicious.
• It pollutes the existing global BA.
• It is very unstable. The result will change each time you run it. In particular, the calculation of the direction of gravity is unstable.
• There seems to be a difference in the optimal strategy for keyframe insertion: compared to Visual SLAM, we needed to insert more keyframes for scale and gravity direction initialization.
• My impression from the experiments so far is that the time taken for initialization is too long. In some applications, it may be necessary to move the camera randomly until it is initialized.

[ymd-stella]

For your purpose, you need to implement only prediction, and may not need initialization and graph optimization (if the gravity component is removed from the IMU input by preprocessing).

[an99990]

would it be possible to filter features coming from moving objects using optical flow ? moving features would be eliminated ? This is a feature that I wan to work on, any thoughts ?

[ymd-stella]

I think the problem is poorly defined. You say remove moving objects, but if you are in a train, what is the moving object? Do you remove the train?

Why would you want to remove the moving object? Do you really need to remove the moving object?

[an99990]

What Id like to do is remove feature belonging to moving objects so it doesnt affect the pose estimation of the camera. like ds-slam is doing: https://arxiv.org/pdf/1809.08379.pdf . features extracted from a person walking by would be removed from the calculation.

[an99990]

from their paper

[ymd-stella]

What Id like to do is remove feature belonging to moving objects so it doesnt affect the pose estimation of the camera.

Have you checked to see what moving objects affect the pose estimation?

If the camera's field of view is wide enough, moving object is ignored by pose optimization or local BA. Then, landmarks that cannot be tracked are removed by local_map_cleaner.

[ymd-stella]

@an99990
Please do not add your question where another person has asked something unless it is the exact same issue. It will send an unwanted notification to the first questioner. Please create a new question and include a reference link.
It will also be harder to get answers because we will not be able to track unanswered questions.