question about implements LRU policy

[xiaoxiangshusheng]

In turbomind.md, turboMind is a memory-pool-liked object that also implements LRU policy so that it can be viewed as a form of cache of KV caches. However, the code in lmdeploy seems to use LFU.

Althrough return blocks_[i].timestamp < blocks_[j].timestamp in code, timestamp is counted by the cumulative form . In other words, timestamp is not a representation of the current time. In code, the timestamp will be added 1 if the blocks is touched. So could you
help me to answer my confusion ablout LRU policy to delete evict blocks?

[lzhangzz]

Even though timestamp is cumulative, blocks_[i].timestamp < blocks_[j].timestamp still implies i is used less recently than j.

Notice that the blocks of a sequence is touched in reversed order. This enforce the blocks in the back of a sequence is evicted before blocks in the front of the sequence.