how to get a 3d point when you have canvas 2d point?

[arunkkarepu]

how to get a 3d point when you have canvas 2d point when you are clicking on any mesh?

[vasiledirla]

converting mouse position into the 3D world position is not just a formula because it depends of the model you are rendering and the position of it in your 3D world.

once you have (x,y) position of your mouse on the canvas then you know this is a point in 2D,
well it is a line in 3D (the line which connects the camera position with the NEAR_PLANE of your 3D world)

after you detect this line (it's equation) you have to test it against your rendered models to check which polygon intersects this line in the nearest point compared to the camera position.

Having the polygon and the line you have to detect the intersection point between these 2 and then you'll know where in the 3D world you clicked. (x ,y, z)

Here's how to do it exactly, step by step.

0. Obtain your mouse coordinates within the client area
1. Get your Projection matrix and View matrix if no Model matrix required.
2. Multiply Projection * View
3. Inverse the results of multiplication
4. Construct a vector4 consisting of
   x = mouseposition.x within a range of window x - transform to values between -1 and 1
   y = mouseposition.y within a range of window y - transform to values between -1 and 1 - remember to invert mouseposition.y if needed
   z = the depth value (i hope out z buffer supports the z value in the interval [-1, 1])

w = 1.0

5. Multiply the vector by inversed matrix created before
6. Divide result vector by it's w component after matrix multiplication ( perspective division )