Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Add allocation boundaries for inline task memory requirements #12

Open
seberg opened this issue Jan 9, 2025 · 0 comments
Open

Comments

@seberg
Copy link
Contributor

seberg commented Jan 9, 2025

gh-11 applies the quick fix to work with the new one-pool implementation of legate. However, not providing any upper bound for the memory allocation needs prevents parallel execution of other tasks.

While it is a bit hard to know if libcudf is doing temporary allocations, at least for tasks such as unary/binary operations this should be easy.
For a task like CSV reading or join/aggregates it is harder since the (current) worst case scenarios for memory use can be surprisingly high, although, maybe even then it is useful to add the estimate. E.g. when reading a small CSV file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant