How to know if there is a cache hit before requesting?

[mayalinetsky-kryon]

After reading the documentation and understanding the behind-the-scenes I got a sense that we can know if there will be a cache hit if* we have all questions ordered chronologically, by using only the embedding function, the similarity evaluation and the post-process function.

Am I correct?
Does GPTCache have a built-in function that does this? If not, how do I know there was a cache hit after sending a request to the LLM?

*[I assume here that the cache is infinite, and no data is removed from it.]

[SimFG]

There is no way, because you also need a component to determine whether two vectors are similar, at least a library like faiss. If you just want to see if it exists in the cache, you can use the encapsulated api method --get, https://github.com/zilliztech/GPTCache/blob/main/gptcache/adapter/api.py#L105

[mayalinetsky-kryon]

I can use the similarity evaluation function to see if two vectors are similar, no?

[SimFG]

The most common similarity evaluation is to use the similarity distance of vectors by vector database or other libs. If you only use cos distance calculation, I feel that this is not very reliable. I have not tried this method. If you switch to other similar evaluations, it is to use models.