-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: Release GIL around C functions #391
Open
jakirkham
wants to merge
8
commits into
rapidsai:branch-0.13
Choose a base branch
from
jakirkham:rel_gil_ucx_c
base: branch-0.13
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Commits on Jan 14, 2020
-
Mark C level functions as
nogil
These functions should be safe to use without the Python GIL as they are not creating or destroying Python objects. So mark them as such. Note: This does **not** actually release the GIL. We still have to do that ourselves. However this allows us to use these in code sections where we have already released the GIL. We can also continue to use them when we have the GIL as well.
Configuration menu - View commit details
-
Copy full SHA for 34b65dc - Browse repository at this point
Copy the full SHA 34b65dcView commit details -
Cython already infers these are `object` type given the assignment type. So these definitions can be dropped without any loss in affect. This will make it easier to move the assignment into the GIL block later.
Configuration menu - View commit details
-
Copy full SHA for a603c54 - Browse repository at this point
Copy the full SHA a603c54View commit details -
Make callbacks
nogil
, but acquire GIL internallyAs these functions are effectively called from `nogil` contexts now given the C functions calling them are `nogil`, make them `nogil` as well, but acquire the GIL internally for the parts of the function that require it. Namely when the Python callback is run and its results are added to the asyncio Future.
Configuration menu - View commit details
-
Copy full SHA for 7d3b137 - Browse repository at this point
Copy the full SHA 7d3b137View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4d97b4a - Browse repository at this point
Copy the full SHA 4d97b4aView commit details -
Configuration menu - View commit details
-
Copy full SHA for d808ba5 - Browse repository at this point
Copy the full SHA d808ba5View commit details -
Release GIL before calling UCX functions
Go ahead and release the GIL before calling UCX functions. They should not be creating, destroying, or modifying any Python objects. Once any of our callbacks are called that do need to manipulate Python objects, we make sure to reacquire the GIL anyways.
Configuration menu - View commit details
-
Copy full SHA for 5cbc5e2 - Browse repository at this point
Copy the full SHA 5cbc5e2View commit details -
As this is the type expected by the C level functions that we call later when using `port`, go ahead and coerce it to `uint16_t` in advance. This should make it easier to release the GIL later on.
Configuration menu - View commit details
-
Copy full SHA for c293849 - Browse repository at this point
Copy the full SHA c293849View commit details
Commits on Jan 25, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 65721dd - Browse repository at this point
Copy the full SHA 65721ddView commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.