-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEA] expose python & C API for prefiltered brute force #174
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this looks great! thanks for getting this in.
some minor comments / feedback below:
/ok to test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm! thanks for this,
/ok to test |
/ok to test |
/ok to test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this API is coming along great! Moslty minor things at this point, but I think this should be ready very soon.
*/ | ||
enum cuvsPrefilterType { | ||
/* No filter */ | ||
NO_FILTER, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We will need a way to specify a custom filter function (maybe to accept a function pointer?) also. While bfknn currently only supports bitmap / bitset, we also need to expose the filtering APIs for the other ANN indexes (most ideally through the same API).
I'm okay pushing this to a future change, since it's going to be needed ASAP so that we can expose the remaining filtering functions for the other algorithm. Can you create an issue for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I will!
/ok to test |
@@ -90,48 +302,167 @@ void recall_eval(T* query_data, | |||
min_recall)); | |||
}; | |||
|
|||
template <typename T, typename IdxT, typename bitmap_t = uint32_t> | |||
void recall_eval_with_filter(T* query_data, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please make sure these are testing all possible combinations of the filter struct/type? Eg NULL, NO_FILTER, BITMAP, and BITSET?
We should set a good precedent for testing these early, so that all follow-on filtering implementations can do the same thing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that makes sense. I just checked the code, the NO_FILTER
, BITMAP
have been covered(pls refer to https://github.com/rapidsai/cuvs/pull/174/files#diff-1b162130023c13d62c06d91f3aab663d943091dddd8a38813c27f65e7d6c61dfR93), and the BITSET
is reserved for other algorithms to compatible which is not be implemented in brute-force. Would you like me to implement BITSET in brute-force and test it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm!
/ok to test |
/ok to test |
/merge |
Authors: - rhdong (https://github.com/rhdong) - Ben Frederickson (https://github.com/benfred) Approvers: - Ben Frederickson (https://github.com/benfred) - Corey J. Nolet (https://github.com/cjnolet) URL: rapidsai#174
No description provided.