The predicted trajectories are switching between left and right when flying towards an obstacle

[coffear]

Both in simulation and real-world experiments, I find that the predicted trajectories will be switching between left and right when the vehicle is flying towards an obstacle. For the neural network, it is natural because it is just trying to avoid the obstacle. The problem is that it keeps this kind of switch, finally leading the vehicle to crash on the obstacle. In another word, when there is an obstacle in front of the quadrotor, the neural network tells the robot to go to the left to avoid the obstacle, so the quadrotor moves to the left, maybe after a second, the neural network asks the robot to go to the right to avoid the obstacle.... then again left, right, left, right......

In your paper and the codes, the processes of fitting by polynomials or projecting the trajectory obtained from the raw predicted trajectory are basically trying to minimize the control input cost. But I do not think this can systematically solve the problem mentioned above.

However, making the decision on which side to go to avoid the obstacle is quite important for an "AI" system. I do not see any clear explanation in the paper, and also failed to do it in the real-world experiments

[antonilo]

Hi! Thanks for your detailed feedback! The switching between modes could be a problem. One way to control it is decreasing the acceptance threshold. In this case more trajectories will pass to the fitting, and the predictions will be more consistent. We make the decision based on energy minimization (we discuss this in the paper). Minimizing energy will bias trajectory to be temporally consistent.
But you can always construct some cases where it will fail. Some recent work proposed new losses to ease this problem.

[coffear]

@antonilo Glad to hear back from you and thanks for your detailed explanation! I will look into your suggestions and try to check them out.

Till now, I further investigated this case, it seems to decrease the acceptance threshold would not improve the performance. I guess the problem might be aroused by: I was implementing a high-speed training model (7 m/s on average) to a low-speed flight (~1 m/s). I found that the behavior of the trajectory prediction model is highly related to the flight behavior of a human pilot when people are creating their own training dataset.

However, as we know, in a practical situation, the flight speed can cover a large range (e.x., 0 to 7 m/s). So the question would be, how to use a single model to deal with this case, not about an average speed. This would be more attractive to practical implementation. Hence this might lead to that if it needs more layers of the network to deal with the different flight speed (the onboard odometry information).

Thank you very much and look forward to your reply! (BTW, this is an awesome work)

[coffear]

@antonilo BTW, do you have a low-speed trained model ( < 2m/s on average ), if you have it, could you share it with me, it could save me a lot of time to verify the problem. My personal PC is not good, and it is not smooth when I am controlling the flight in obtaining the training data

[LeiOuYang]

@coffear I also want to reproduce this project recently, but I don't know how to build a real environment. The project seems to only mention how to simulate and train. Do you have more detailed instructions on how to build and use the real machine? Thanks

[antonilo]

You can get instructions on how to build and control the drone here: https://github.com/uzh-rpg/agilicious