Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wrong keypoint coordinates #12

Open
christian-rauch opened this issue Sep 15, 2020 · 7 comments · May be fixed by #14
Open

wrong keypoint coordinates #12

christian-rauch opened this issue Sep 15, 2020 · 7 comments · May be fixed by #14

Comments

@christian-rauch
Copy link

The keypoint coordinates are extracted from the heatmap via:

xs, ys = np.where(heatmap >= self.conf_thresh) # Confidence threshold.
if len(xs) == 0:
return np.zeros((3, 0)), None, None
pts = np.zeros((3, len(xs))) # Populate point data sized 3xN.
pts[0, :] = ys
pts[1, :] = xs
pts[2, :] = heatmap[xs, ys]

The indices returned by np.where, respectively np.nonzero are in order of the dimensions. In a NumPy array, the first dimension is along the rows (i.e. the y coordinates in an image), the second dimension is along the columns (i.e. the x coordinates in an image).

Hence, syntactically this should be:

ys, xs = np.where(heatmap >= self.conf_thresh) # Confidence threshold.
# [...]
pts[0, :] = xs
pts[1, :] = ys
pts[2, :] = heatmap[ys, xs]

The actual variable name does not matter later, as those values are correctly normalised.

@ddetone
Copy link

ddetone commented Sep 22, 2020

Good catch! I think the variable names are wrong, but the output should be correct.

Unfortunately I don't have write privileges and can't update the code :/

@christian-rauch
Copy link
Author

Yes, the output is not affected since the inverted meaning is inverted again when accessing the heatmap and for the grid_sample.

When porting this to other languages, e.g. C++, you have to sample the feature map with (x,y) tuples. The python code currently suggests that these should be (y,x) tuples, resulting in the wrong descriptors.

@laisimiao
Copy link

@christian-rauch I try a small example, and I feel no wrong:

import numpy as np
a = np.random.randn(3, 3)
a
Out[4]: 
array([[ 0.24942095,  0.8585088 , -0.1106783 ],
       [ 0.07011566,  0.21370989,  0.81164736],
       [ 2.40070676, -0.11000709, -1.76859847]])
np.where(a > 0)
Out[5]: (array([0, 0, 1, 1, 1, 2]), array([0, 1, 0, 1, 2, 0]))

Where do I understand wrongly?
np.where return (x, y)

@christian-rauch
Copy link
Author

Where do I understand wrongly?
np.where return (x, y)

The first row in your matrix a is indeed the first column of the image. That means an index in your first matrix row is the index of the first image column, which is 'y'.

You will see this better when you visualise the data:

import numpy as np
import matplotlib.pyplot as plt

if __name__ == "__main__":
    a = np.zeros(shape=(5,7))

    # set pixel at x=5 and y=2 to 1
    a[2,5] = 1

    # find the coordinate of value '1'
    y,x = np.where(a==1)
    print("point (x,y): ",x[0],",",y[0])
    plt.imshow(a)
    plt.show()

The image is:
test

And the 1 (yellow square) is located at:

point (x,y):  5 , 2

@laisimiao
Copy link

@christian-rauch Oh I got it, thanks for your patient and vivid explanation.

BTW, could you provide a script that can draw Qualitative Results of a pair of images like paper show(see followiing)? Thank you for your help
image

@christian-rauch
Copy link
Author

christian-rauch commented Sep 23, 2020

@laisimiao I opened pull request #14 to fix this issue. Is this sufficient to demonstrate that it works?

Btw, at

keepy, keepx = np.where(grid==-1)

this was already done correctly.

@Yvaine
Copy link

Yvaine commented Oct 12, 2020

Hi, I think @christian-rauch is right, and should fix, because
samp_pts[0, :] = (samp_pts[0, :] / (float(W)/2.)) - 1.
the samp_pts[0] represent x axis

@christian-rauch christian-rauch linked a pull request Jan 27, 2021 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants