Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accuarcy #10

Open
Wu-ZW opened this issue Sep 3, 2024 · 18 comments
Open

Accuarcy #10

Wu-ZW opened this issue Sep 3, 2024 · 18 comments

Comments

@Wu-ZW
Copy link

Wu-ZW commented Sep 3, 2024

I run this repo, dataset is Mountain. got abs_res.txt then using evo_traj to evaluate traj
image

there is a big difference between trajs, the parameters I set is wrong?

@vasnakh
Copy link

vasnakh commented Sep 11, 2024

I am also getting a large discrepancy for Gravel terrain. The shape of the path is not as crazy but seems to have large offset as you can see in the pic.
Gravel

@LuoXubo any clue why this is happening? I am guessing there needs to be some values that needs to be changed in order to set the initial position to correct location for each terrain maybe? I'm not really sure and need to investigate more about this honestly.

@Wu-ZW
Copy link
Author

Wu-ZW commented Sep 13, 2024

I am also getting a large discrepancy for Gravel terrain. The shape of the path is not as crazy but seems to have large offset as you can see in the pic. Gravel

@LuoXubo any clue why this is happening? I am guessing there needs to be some values that needs to be changed in order to set the initial position to correct location for each terrain maybe? I'm not really sure and need to investigate more about this honestly.

I also think that some parameters need to be changed.And this repo is not design for real-time.

@Wu-ZW
Copy link
Author

Wu-ZW commented Sep 13, 2024

I am also getting a large discrepancy for Gravel terrain. The shape of the path is not as crazy but seems to have large offset as you can see in the pic. Gravel

@LuoXubo any clue why this is happening? I am guessing there needs to be some values that needs to be changed in order to set the initial position to correct location for each terrain maybe? I'm not really sure and need to investigate more about this honestly.

Do you read this code of repo? I am also read this repo for better localization performance.If so, could we discuss perspectives by email?

@LuoXubo
Copy link
Owner

LuoXubo commented Sep 13, 2024

I am also getting a large discrepancy for Gravel terrain. The shape of the path is not as crazy but seems to have large offset as you can see in the pic. Gravel

@LuoXubo any clue why this is happening? I am guessing there needs to be some values that needs to be changed in order to set the initial position to correct location for each terrain maybe? I'm not really sure and need to investigate more about this honestly.

@vasnakh Hi, you can think of it as requiring a rotation and translation matrix from image coordinates to world coordinates, so you need to use these matrices to make the two trajectories coincide.

@LuoXubo
Copy link
Owner

LuoXubo commented Sep 13, 2024

I run this repo, dataset is Mountain. got abs_res.txt then using evo_traj to evaluate traj image

there is a big difference between trajs, the parameters I set is wrong?

@Wu-ZW The trajectory looks too uneven. Can you provide more configuration information? For example, dataset and hyperparameter settings, etc.

@Wu-ZW
Copy link
Author

Wu-ZW commented Sep 13, 2024

  1. server.sh
    image
    2.relloc.sh
    image
    3.absloc.sh
    image
    I do not change other parameters..

@Wu-ZW
Copy link
Author

Wu-ZW commented Sep 13, 2024

  1. server.sh
    image
    2.relloc.sh
    image
    3.absloc.sh
    image
    I do not change other parameters..

I confused the evalation method.. evo_ape A.txt B.txt. A.txt means gt.txt and B.txt means abs_res.txt?
image
But the abs_res.txt is not the fused postion right? @LuoXubo

@hitersyw
Copy link

I am also getting a large discrepancy for Gravel terrain. The shape of the path is not as crazy but seems to have large offset as you can see in the pic. Gravel

@LuoXubo any clue why this is happening? I am guessing there needs to be some values that needs to be changed in order to set the initial position to correct location for each terrain maybe? I'm not really sure and need to investigate more about this honestly.

I have got the same results, have you solved the problem? thanks
@vasnakh

@hitersyw
Copy link

I am also getting a large discrepancy for Gravel terrain. The shape of the path is not as crazy but seems to have large offset as you can see in the pic. Gravel
@LuoXubo any clue why this is happening? I am guessing there needs to be some values that needs to be changed in order to set the initial position to correct location for each terrain maybe? I'm not really sure and need to investigate more about this honestly.

I have got the same results, have you solved the problem? thanks @vasnakh

@LuoXubo

@hitersyw
Copy link

I am also getting a large discrepancy for Gravel terrain. The shape of the path is not as crazy but seems to have large offset as you can see in the pic. Gravel
@LuoXubo any clue why this is happening? I am guessing there needs to be some values that needs to be changed in order to set the initial position to correct location for each terrain maybe? I'm not really sure and need to investigate more about this honestly.

Do you read this code of repo? I am also read this repo for better localization performance.If so, could we discuss perspectives by email?

could we discuss perspectives by email? [email protected]
@Wu-ZW

@LuoXubo
Copy link
Owner

LuoXubo commented Nov 18, 2024

Hi, the performance of JointLoc relies on the prior location of the UAV. Please make sure you set the right prior map index corresponding to the landscape. Like 79 for Crater, 61 for Gravel and 107 for Mountains.

@hitersyw
Copy link

Hi, the performance of JointLoc relies on the prior location of the UAV. Please make sure you set the right prior map index corresponding to the landscape. Like 79 for Crater, 61 for Gravel and 107 for Mountains.

thanks for your kindly reply, i have check the prior map index, i have tested two datasets, the results are all same to represent the results,
屏幕截图 2024-11-18 191421
屏幕截图 2024-11-18 220955

@LuoXubo
Copy link
Owner

LuoXubo commented Nov 20, 2024

Hello, the reason why the two sets of trajectories do not overlap is that the generated trajectories are still in the pixel coordinate system, and a set of RT matrices are needed to convert the trajectories to the world coordinate system.

@vasnakh
Copy link

vasnakh commented Dec 11, 2024

@hitersyw @Wu-ZW my email address is [email protected] if you'd like to discuss this via email. Feel free to shoot me an email so we can start the conversation. Sorry, had to work on another project and didn't have time to dig deep into this and now have time to get back to it.

@vasnakh
Copy link

vasnakh commented Dec 11, 2024

Hello, the reason why the two sets of trajectories do not overlap is that the generated trajectories are still in the pixel coordinate system, and a set of RT matrices are needed to convert the trajectories to the world coordinate system.

  1. So you'd have to basically pick few time synced points (at least 3) from both gt.txt and CompleteTrajectory.txt and use something like Kabsch–Umeyama algorithm to find the RT and scale? How did you approach this for Crater terrain?

  2. Is it possible to also fuse IMU for relative pose (by leverage ORB-SLAM2)?

@LuoXubo
Copy link
Owner

LuoXubo commented Dec 12, 2024

Hi, you can check the commit to fix this problem. I've updated the alignment in System.cc.

@vasnakh
Copy link

vasnakh commented Dec 12, 2024

Hi, you can check the commit to fix this problem. I've updated the alignment in System.cc.

Thank you for quick reply. Based on my observation the code still does not give good results for any other terrain (meaning they are not aligned with reference/gt). I only see correct alignment for Crater terrain. I see that you have R,T, and s defined in AbsLoc/align.py and believe this is part of aligning the trajectory (at least for absolute pose)

@vasnakh
Copy link

vasnakh commented Dec 13, 2024

For Gravel terrain, this was my approach to get R,T, and s (which can be applied to other terrains as well):

  1. I took in AbsLoc/align.py i took out align function and just used pure resX, resY, and z.
  2. ran the code, got abs_res.txt
  3. using abs_res.txt and gt.txt, i synced the time and took 3 random points (first, middle, and last index) and then used Kabsch–Umeyama to get R,T, s

Results are:
R = np.array([[-0.026167361061090858, -0.9996575752618211, 3.7904555208493894e-05],
[0.9996571810683438, -0.026167384408796273, -0.0008878804510469469],
[0.0008885682818829484, 1.4658072467742674e-05, 0.9999996051156965]])
T = np.array([106.31458835805707, -108.61389819267589, -18.931387333713744])
s = 0.05530914856783145

Now putting these values in AbsLoc/align.py gives me a more aligned result with rmse 0.613518 when using CompleteTrajectory.txt

@LuoXubo is R,T and s for Gravel terrain similar as above for you? Also, would really appreciate it if you let me know how you actually calculated R,T, and s on your end. Did you also use gt.txt and abs_res.txt?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants
@vasnakh @hitersyw @LuoXubo @Wu-ZW and others