Find vectors that are not similar

[Go-MinSeong]

I don't want to find similar vectors, I want to perform the process of finding dissimilar vectors.
What I want to do is select data through an iterative process of randomly selecting one vector to use as a query vector, quickly finding vectors that are not similar, and then using that vector as the query vector.

For example, if I am selecting 30,000 out of 100,000, I have organized my collection of 100,000 into 10 partitions of 10,000, and when I run the search, I am giving an offset of 8000 and a limit of 2000 to find vectors that are not similar.

I'm wondering if the way I've organized it is appropriate and if there are any improvements I can make.
Also, does milvus have a way of finding non-similar vectors instead of similar ones?

Thanks for reading the post and I look forward to your answer.

[yhmo]

Milvus doesn't have any interface to "search non-similar vectors".

"offset" is a workaround. Set limit=10000 and offset=8000

[yhmo]

By the way, the index type should be "FLAT" to ensure all the vectors are computed.

[xiaofan-luan]

I don't want to find similar vectors, I want to perform the process of finding dissimilar vectors. What I want to do is select data through an iterative process of randomly selecting one vector to use as a query vector, quickly finding vectors that are not similar, and then using that vector as the query vector.

For example, if I am selecting 30,000 out of 100,000, I have organized my collection of 100,000 into 10 partitions of 10,000, and when I run the search, I am giving an offset of 8000 and a limit of 2000 to find vectors that are not similar.

I'm wondering if the way I've organized it is appropriate and if there are any improvements I can make. Also, does milvus have a way of finding non-similar vectors instead of similar ones?

Thanks for reading the post and I look forward to your answer.

can you give a use case for find dissimilar vectors?

[Go-MinSeong]

I want to proceed with data selection through an iterative process of selecting a non-similar vector as the query vector and doing the same with the selected vector.

The idea is to represent the space of all the data with a subset of the data.