Recovering intrinsic camera matrix from simulation camera

[sunshineinsandiego]

Hi - I am trying to recover an homogenous intrinsic camera matrix of the form:

$$\begin{bmatrix} f_x & 0 & c_x & 0\\\ 0 & f_y & c_y & 0\\\ 0 & 0 & 1 & 0 \end{bmatrix}$$

From the simulator camera specifications here. @adnanmunawar noted here that I should be able to recover the intrinsic matrix given the near plane, far plane, FOV, and aspect ratio, and I would like to confirm that I am doing this correctly.

Using this source and this source, it appears that the conversion from simulation camera parameters to OpenGL projection matrix is:

$$\begin{bmatrix} e & 0 & 0 & 0\\\ 0 & e/a & 0 & 0\\\ 0 & 0 & -(f+n)/(f-n) & -2fn/(f-n)\\\ 0 & 0 & -1 & 0 \end{bmatrix}$$

Any advice on how to go from this projection matrix to the camera intrinsic matrix? This seems like another possible step, but not sure.

Thanks for your help!

[adnanmunawar]

Hello, have you taken a look at this document by @jabarragann ?

[sunshineinsandiego]

Ah yes! I have, but when I test it by constructing the simulator frustum and the opencv camera intrinsic matrix, then projecting an identical point in 3D space onto the image plane, I get different answers.

But if that conversion in the doc is correct, perhaps the error in projection is due to something other than the mapping between the frustum vs. K matrix?

[adnanmunawar]

Hmm, it is possible. Let's see what @jabarragann thinks.

[jabarragann]

Hi @sunshineinsandiego,

Apologies for the late reply. In this document, we projected some 3D points specified in the needle reference frame and projected them back into the image using opencv, so I am confident that the formulas presented here are correct. If you describe your setup and the problems you are facing, I am more than happy to help you debug the problem.

[sunshineinsandiego]

Thanks @jabarragann. Sounds like you ran the tests that I'm trying to tie out. I think the issue that I was having was related to the external camera matrix parameters I was setting. Through a bit of trial and error it seems to be fixed now, but I thank you for your offer to help! Will close this issue.

[jabarragann]

Fantastic, I am glad it got fixed!