-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I'd like to contact some one for discus eventual winndows port ... #2
Comments
Hi, I think I use about the same approach as what you depicted. I call it user-space mmap() implementation. Basically, the ljmm hooks to luajit by replacing its function call the mmap/munmap/mremap. At start, the ljmm is to reserver the address space of [program's BSS, 2G], or [1G, 2G](so the glibc's mmap take over the [BSS, 1G] part). I use a buddy system to keep track of the free blocks. Buddy system is able to combine adjacent with 2n pages into a bigger block (with 2(n+1) pages). While buddy allocator can handle external fragmentation very well, it's not able to handle internal fragmentation. In the worst, extreme case, the infragmentation would incur 50% waste. But in practice, it seems we should not worry about it. I try to improve locality (by reducing the working set), but so far I don't have any good solution. I don't have lots lua code at hand, I tested this work by hooking the ljmm to C/C++ applications. I measure the peformanc impact using CPU2000int, and did not find any negative performance impact. The testing is done by mmap() is small block right after .bss when app starts); this way glibc's malloc() is done by mmap()ing a big chunk of memory, and then carve an block out of it). C/C++ application test mmap/munmap() very well. However, the mremap() is not sufficiently tested. Unforuantely, mremap() is used extensively in luajit. |
hi, sure, i can give all code, what is not dependent on our, internals(i use our logging framework, but except that all is free). what surprised me, is a fact to need handle free blocks (buddy blocks management in your case) and some sort of allocation it self, if luajit handle it already in lj_alloc.c.
when lua ask me some ram block for size, i just run in my bit arrays, linearly and find first place where are enough not used pages for that block, then i flag pages, and record block, so i can free it later.
i choose this approach cause im injecting my code at page level , not at malloc/free level sorry for my english, but i hope i explain me well, so we can continue our discussion. ladislav ;) Inviato da iPad
|
Hi,
I have bunch On 04/18/2015 09:10 AM, LadislavSopko wrote:
We once tried to modify Linux kernel such that mmap() such that it can
I tried to use bitmask as well. However, I don't know how to deal with If the luajit keep allocating small objects, and hence mmap-ping 32kb The only reason I went for buddy allocation is to address the For us, Internal fragmentation is not a big deal as most object are
Do you try to merge adjacent blocks into a big one?
If the released block have 56kb in size, and the next allocation is Also, while luajit see a virtually released block, the OS view it as Linux's madvise() comes in handy for telling OS not to swap particular Does Window have similar problem? Best Regards! |
hello again :), I'm really happy to have possibility, to discuss, ill leave comments in Inviato da iPad
yes this should be a problem, yes lj use 31 bit wide addresses, ill will handle this.
have to test , what happen when i unprotect page, im in early proof of concept stage.
|
Hello, I have some new results of testing. I found that luajit allocator ask for ladislav. 2015-04-19 23:33 GMT+02:00 Ladislav Sopko [email protected]:
|
I manage it to work, In linux it is even more fast as original luajit. In windows it loose some farction regards |
That is truly great! I will read your code sometime this week-end or Best Regards On 04/29/2015 09:00 AM, LadislavSopko wrote:
|
Thank you :) |
I did small change, Now it can use all 32bits so actually it reserve 3,5GB 2015-04-29 18:37 GMT+02:00 yangshuxin [email protected]:
|
How did you manage to sidestep the pointer-sign-extension problem? Thanks On 04/30/2015 05:09 AM, Ladislav Sopko wrote:
|
I don't have manage it, but I found that LJ it self have problem inside jited parts and gc with sign extension, so it's better to hold addresses in max31bits. so I'll limit it back to 31 bit. Its just questin change 2 defines.... Pity, no problem for me, any way, cause our scripts, are relatively small, and 2gb are far enough. Inviato da iPad
|
I'm developing similar solution, but I choose much more simpler
aproach, I just replace mmap/VirtualAlloc calls in lj_alloc.c
with mine cooked mmap/VirtualAlloc calls . My code at start reserve 1-2GB of low address space
and then i just give it, in use to luajit ( I let luajit use it's memory allocator, I just provide blocks of address pages on demand from previously reserved space using RESERVE/COMMIT ram trick in win and similar aproach in nix.). I'd like to discuss some aspects of mine solution in respect of
yours, maby we can share experiences.
regards Ladisalv.
The text was updated successfully, but these errors were encountered: