Does memray monitor GPU memory usage?

[Neronjust2017]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

I' m using memray to monitor my PyTorch model training and I found model inference step allocated most memory as you can see below. However, the inference is on GPU for my model, why did it cost so much memory? Does this frame include GPU memory usage? Also, I monitored GPU usage in my code and it's max value is ~4GB.

Expected Behavior

No response

Steps To Reproduce

No

Memray Version

1.15.0

Python Version

3.10

Operating System

Linux

Anything else?

No response

[godlygeek]

No, Memray does not currently monitor GPU memory usage. It currently monitors allocations and deallocations made using these functions:

• malloc
• free
• calloc
• realloc
• valloc
• posix_memalign
• aligned_alloc
• mmap
• munmap

[Neronjust2017]

No, Memray does not currently monitor GPU memory usage. It currently monitors allocations and deallocations made using these functions:

• malloc
• free
• calloc
• realloc
• valloc
• posix_memalign
• aligned_alloc
• mmap
• munmap

thanks!

what does 18.4GB means? the total memory allocated by this function? or the peak memory usage?
And how can i know the function (1. malloc, free, calloc, etc. 2. the functions in my code) proportion of memory usage at each timestamp in the below figure?

[godlygeek]

what does 18.4GB means? the total memory allocated by this function? or the peak memory usage?

The total amount of memory allocated by that function and not yet freed at the moment in time when the process reached its peak memory usage. See https://bloomberg.github.io/memray/flamegraph.html#interpreting-flame-graphs

And how can i know the function (1. malloc, free, calloc, etc. 2. the functions in my code) proportion of memory usage at each timestamp in the below figure?

The default flame graphs only show you what stack allocated each chunk of memory at the moment in time when the process reached its peak memory usage. You can run with memray flamegraph --temporal to generate a flame graph that allows you to investigate other moments in time. See https://bloomberg.github.io/memray/flamegraph.html#temporal-flame-graphs

Our flame graphs don't distinguish between allocations from different allocators, but memray stats shows you the number of allocations made by each allocator.