-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multiprocess online deployment #331
base: main
Are you sure you want to change the base?
Conversation
0fe8745
to
fba485f
Compare
@EthanMarx I think this is ready for a review. I'm running some benchmarking on what's currently in |
This PR adds multiprocessing to our online deployment function. The main constraints when setting this up were:
So, with this setup, everything on GPU happens in the same process. The flow of information goes as follows:
Process 0: Main Aframe loop
Note: using a shared memory object because passing a large tensor between processes introduces latency
Process 1: Event processing
graceid
graceid
to AMPLFI and p_astro queuesProcess 2: Skymap creation
graceid
Note: technically, the
graceid
could be received first, which is why there's a check for that. Should be a way to do this more cleanly, but I think this is fine for now.Process 3: p_astro calculation
graceid
@EthanMarx There's a more to do on this, like writing the buffers to disk and probably better logging with all of the different processes happening, but it would be good to get some eyes on it now.