Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

strip subcommand runs out of memory #113

Open
Zannick opened this issue Jul 13, 2023 · 0 comments
Open

strip subcommand runs out of memory #113

Zannick opened this issue Jul 13, 2023 · 0 comments

Comments

@Zannick
Copy link

Zannick commented Jul 13, 2023

I'm using bytehound to try to pinpoint what in my program is allocating or reserving the large amounts of memory that cause my program to crash. Previously, #21 and #18 recommended the "squeeze" subcommand (which is now "strip"), and that was working for me for some time on files of size 700GiB+ before I began setting MEMORY_PROFILER_CULL_TEMPORARY_ALLOCATIONS=1 and occasionally MEMORY_PROFILER_TEMPORARY_ALLOCATION_LIFETIME_THRESHOLD=100000 or thereabouts when running my program through bytehound.

I have 48 GiB of RAM on my system. An 11 GiB data file was generated when I ran my program with bytehound and set MEMORY_PROFILER_CULL_TEMPORARY_ALLOCATIONS=1 MEMORY_PROFILER_TEMPORARY_ALLOCATION_LIFETIME_THRESHOLD=50000; I killed the program via SIGINT when it reached 93% memory usage. Running this file through bytehound server doesn't work due to OOM. Surprisingly, running this file through bytehound strip is also causing an OOM. Via top I can watch the virtual memory space increase to 111GiB whereas the program only reached 84GiB virtual.

Do I need to set the same parameters when running strip? Or is there some other reason that it runs out of memory?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant