CPUmilvus 0.10.2 CPU resources are not released immediately after the comparison operation

[Haohao966]

Stress test, call 100 times in a row for feature comparison
Sleep for 1 second between each test alignment, The average time spent was lower than for people who didn't get any sleep
Is this because CPU resources are not released immediately after each alignment test, which affects the performance of subsequent alignment tests

[shiyu22]

Milvus loads vector data or index into memory according to the cache_size parameter, and Milvus does not release memory.

So it's not normal for worse performance to occur with 1-second sleep, can you describe it specifically? And how much data do you have? And what is the cache_size parameter?

[lllxxx3030]

Stress test, call 100 times in a row for feature comparison, data size is 1 million:
Sleep for 1 second between each test alignment, The average time spent 200ms, CPU utilization about 60%
Don't sleep between each test alignment, The average time spent 600ms, CPU utilization about 100%

Is it related to not releasing CPU resources？

[yhmo]

You can refer this article: https://zhuanlan.zhihu.com/p/148274838
All search request are put into a queue, processed one by one. For requests with same parameters, milvus could combine multiple requests into one request to execute. Before current request is finished, all other requests will be pending and waiting.

If you sent 100 search requests at same time, milvus will combine some of them to be one request. Assume each request time cost 600 ms, and milvus combine 30 requests into one, the combined request time cost is less than 30x600 ms, maybe 30x200 ms, so you see the average time is 200 ms.

But if each request sleep for 1 second, milvus won't combine them, so the average time is 600 ms.