Add kv state cache module.#1777

Merged

sighingnow merged 20 commits intomainfrom kv-state-cache

Mar 1, 2024

+6,430-19

Commits on Feb 28, 2024

Support kv state cache for llm using vineyard (#1720 )

vegetableysm
and
dashanji
committed
Improve the split logic and implement the serialization and deserialization of radix subtree. (#1730 )

dashanji
authored and
vegetableysm
committed
Refactor code of radix split subtree and fix bug of radix split. (#1735 )
vegetableysm
committed
Support merge tree / merge object / bug fix. (#1738 )
vegetableysm
committed
Support merge cache object with radix tree. (#1741 )
vegetableysm
committed
Replace lz4 with zstd (#1744 )

dashanji
authored and
vegetableysm
committed
Recycle resources of kv state cache object. (#1748 )
vegetableysm
committed
Add the unit test for the radix tree. (#1749 )

dashanji
authored and
vegetableysm
committed
Fix the bug in the radix split logic. (#1751 )

dashanji
authored and
vegetableysm
committed
Support layer for kv state cache (#1766 )
vegetableysm
committed
Support to store more than 64 entries for kv cache block (#1769 )
vegetableysm
committed
Add ci test for kv state cache and fix bug (#1772 )
vegetableysm
committed
Disable code formatting of radix tree and fix a bug of serialization and deserialization (#1771 )

dashanji
authored and
vegetableysm
committed
Add a benchmark test for kv state cache (#1774 )

dashanji
authored and
vegetableysm
committed
Add the rax linsence and fix a warning (#1778 )
dashanji
authored
Clean code.
vegetableysm
committed
Move llm tests to llm-cache directory
vegetableysm
committed

Commits on Feb 29, 2024