feature request, mmap option

[ouvaa]

@phuslu pls pls pls implement a mmap option that can be shared across golang inter process.
can buy u a few coffees for this. possible to get it done?

maybe with an option to flush to mmap file at regular intervals or something. most importantly so can be used across different separate programs

[phuslu]

I thought it before, here is my roudmap

1. provide a new type cache, e.g. *lru.BytesCache, the key&value both are []byte
2. use mmap/memfd to convert it to offheap
3. make it shareable across golang processes.

[phuslu]

So as we see, I'm reinventing another freecache/fastcache, it seems that a bit sad.

[ouvaa]

@phuslu can you implement it? i know u can make this the fastest for interprocess sharing.

this will be mmapped for inter process golang accessing, which is not available in other solutions

[ouvaa]

@phuslu pls implement it. this will make it work around all the 192 cores issues and more.

[ouvaa]

@phuslu when can you get it done? how can i show my appreciation and thanks for this?

[phuslu]

The reason I haven't start it yet is that I don't have the confidence to do it better than fastcache.

But under your encouragement, let me try reluctantly.

[ouvaa]

@phuslu if you do an mmap version, u already done better than the rest as it'll be the only lru cache that can be accessed across independent and separate processes. please post incomplete code or where u think i can help contribute coz i think i can help u make it faster then.

dont worry about speed for the moment, it's the only mmap lru that is shareable across process.

can u pls start it and i'll help with speeding it up.

[ouvaa]

@phuslu chatgpt may help u out too in certain areas.

[phuslu]

ok, let me push a drafted "BytesCache" commit to poc our ideas today

[ouvaa]

@phuslu i'm not sure how u will have a global mutex to sync for separate golang processes to access it. not sure if u will hit any issues but i have solution. u may come up with a better global mutex lock or similar.

[ouvaa]

@phuslu this is my last bit of performance issue with golang, so i'll be around 24/7 just looking at this issue. i was working on chatgpt with an mmap lru and it's terrible.

i know u definitely will come up with "the best".
seriously will be using across everything, i will refactor all repo and code to use this.

[phuslu]

Found another similar implementation in c/php, https://github.com/happyfish100/libshmcache

The noticeable feature is that it also choose FIFO rather than LRU, this same as fastcache. I think that because of "lock contention" reduction

[ouvaa]

dont worry about them, get it done on your version. in fact, just mapped the variables to the memory mapped memory file e.g. /dev/shm/shm.mmap or something.

as i mentioned, if u found issue with lock, i have solution

[ouvaa]

you can do what u think is best first, i will try the lock issue on my side and send you the code once i verify it can be resolved.

by the way, i think u'll confuse yourself more if you reference others' implementation, that c repo is irrelevant.

your lru will work directly with mmap referencing directly, the only issue u have will be the global lock that's all.

dont need to do the fifo etc and no need to flush writes actually. i've tested mmap file sharing and without flushing it works too as long as you dont reboot the server of course.

p.s. : i mean directly u can just implement a global lock on your lru without any issue. dont do fifo etc. use your version exactly.

2 things need to change only.

1. memory section use mmap file
2. global mutex lock implementation

that's all really.

[ouvaa]

@phuslu do send the code of what u have done so i can try the lock issue if u think i can do better

[phuslu]

1. provide a new type cache, e.g. *lru.BytesCache, the key&value both are []byte

I done the first easiest step.

2. use mmap/memfd to convert it to offheap

I think I need "steal" some fastcache codes.

3. make it shareable across golang processes.

need learn bytedance codes and your guide/help.

[ouvaa]

@phuslu fastcache does not have the things u need, u can reference some code from here:
https://github.com/viant/scache

but this does not work across processes because it is missing somethings. i'm sure u'll figure out.

[ouvaa]

@phuslu check the

case MemoryMappedFileExample:
		cache, err = scache.New(&scache.Config{SizeMb: 256, Location: "/tmp/data.sch"})
		//or 
		cache, err = scache.NewMmapCache("/tmp/data.sch", 256, 0, 0)

i prefer the 2nd option as it's "better" also, please make it in bytes so it's cleaner. the 256 is in megabytes, u can do 256000000 as input parameter. the other two 0s can be ignored for now.

[ouvaa]

@phuslu hope u have seen the scache. it's really easy concept.

[ouvaa]

@phuslu i saw the mmap_file entry now. u picked up really quick.

do mention if u see any issues, pls map the []bytes to the mmap file (and then put a global lock and it's done.)

do show any sort of naive implementation too. i will help add code to some sections as add on feature.

[ouvaa]

@phuslu any issues?

[phuslu]

sorry for late reply and thanks for help, I've been a little busy these days, so I haven't been thinking and developing this feature fully.

Currently I'm blocked in how alloc & place mmapnode into the mmaped file/memory. FIFO algothrim works like a ringbuffer, just alloc & place nodes in the FIFO queue tail then evict the nodes in the FIFO head if needed.

But LRU is more complicated, when it evict old nodes, it will leave the spare holes/segments and wait be recyle.

Maybe I needed implement the linux Slab allocation

[ouvaa]

@phuslu your comment reminds me of this, which is written in c without lru but does have memory "gc".
https://github.com/simonhf/sharedhashfile

just for reference but maybe you can get some inspiration from there. not sure if "gc" on the holes is faster or otherwise.

[ouvaa]

@phuslu is there any issue with using mmap directly against the lru currently? the spare holes etc are acceptable at this stage. scache will use 2x the memory and not really lru but the spare hole issue you have is mostly an edge case issue i guess.

would love to use your mmap version asap as i can help add features to it etc.

do u think u can have an eta on this mmap version? global mutex wise i can help resolve too

[phuslu]

I'm learning fastcache architect, a introduction here(Chinese version) https://www.cnblogs.com/ahfuzhang/p/15840313.html
I think my idea/design will be finialed soon.

do u think u can have an eta on this mmap version?

Sorry, I cannot give it. From experience, I usually need a bit long vacation alone to complete this kind of challenge that requires both mental and physical strength. This time I guess it might be during May 1st the International Labor Day.

[ouvaa]

@phuslu hmm... ok. i'll be looking at this issue daily. if u have any areas need me to help test / improve do mention.

[sprappcom]

@phuslu i have the solution for a high speed lock at > 15mil lock/inlock per second.

can u guesstimate an eta for the mmap implementation?

[phuslu]

Sorry these days I spent most time to improve "phuslu/log" and released its v1.0.93 tag. Today I will turn to this lru library.
I can work out a first draft until this sunday.

[phuslu]

sorry, still spend time for improve github.com/phuslu/log performance. will start lru mmap next day.

[sprappcom]

@phuslu ok. when do u think an eta can be provided for it? just curious on an eta that's all.

p.s. : not rushing u, just very anxious to know that's all.

[sprappcom]

@phuslu can u pls help make sure it's not like using a hash function which will result in lower hit ratio? i would prefer if u can help with having something along the lines of map[string]string etc so that there can be no hash collision.

i've tried implementing something myself but it's not working but i can help with areas if u have any issues too.

[sprappcom]

@phuslu if u need help with test etc. do mention