Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adds optimizations to the UCX DTL to improve performance #58

Closed
wants to merge 25 commits into from
Closed

Adds optimizations to the UCX DTL to improve performance #58

wants to merge 25 commits into from

Conversation

ilumsden
Copy link
Collaborator

This PR adds optimizations to the UCX DTL to improve DYAD's performance. Currently, the optimizations that will be added are:

  1. Allocating memory buffers using UCX
  2. Running a warmup communication with self to help initialize UCX components
  3. Cache UCX endpoints in server to minimize the number of connection establishments required

@ilumsden ilumsden added the enhancement New feature or request label Dec 13, 2023
@ilumsden ilumsden self-assigned this Dec 13, 2023
@ilumsden
Copy link
Collaborator Author

Note that this PR is based on #56 to allow us to use the most recently updated CI for automated testing

@ilumsden ilumsden force-pushed the ucx_opts branch 2 times, most recently from 39a44f8 to 0c39407 Compare December 21, 2023 18:53
@ilumsden ilumsden force-pushed the ucx_opts branch 2 times, most recently from e747495 to ad94370 Compare January 2, 2024 22:48
ilumsden and others added 23 commits January 4, 2024 11:55
@ilumsden
Initial work on converting DYAD from autotools to CMake
d04c6e4
@ilumsden
Updates compile_test.yaml to use CMake instead of Autotools
f93d032
@ilumsden
Typo fix
4f89b70
@ilumsden
Removes "VERSION" from find_package since it's not actually correct
a7b1c05
@hariharan-devarajan @ilumsden
refactored and initial changes to build.
fbec6e7
@hariharan-devarajan @ilumsden
fixes for installation.
ef55339
@hariharan-devarajan @ilumsden
fixed streaming apis.
3863192
@hariharan-devarajan @ilumsden
fixed installation script.
6ed9b1b
@hariharan-devarajan @ilumsden
fixed CI
602c4ae
@hariharan-devarajan @ilumsden
fixed CI
41d5994
@hariharan-devarajan @ilumsden
fix ucx compile
891f24c
@hariharan-devarajan @ilumsden
fix ci
cd5d11b
@hariharan-devarajan @ilumsden
separate setup of spack
dd4e382
@hariharan-devarajan @ilumsden
fix comments
c2f76ba
@hariharan-devarajan @ilumsden
fixed spack script
cec789a
@hariharan-devarajan @ilumsden
fixed python
d6a1678
@hariharan-devarajan @ilumsden
Update compile_test.yaml
c864149
@hariharan-devarajan @ilumsden
Update compile_test.yaml
d127da9
@hariharan-devarajan @ilumsden
Update compile_test.yaml
8259937
@hariharan-devarajan @ilumsden
Changes to public header and libnames
25f75f1 
1. Removed dyad_code from public headers
2. used PROJECT_NAME variable for all code
3. prefixed PROJECT_NAME for util code to avoid conflicts
@ilumsden
Adds Caliper annotations and adds flux-optparse support for server ar…
8574ac8 
…gument parsing
@ilumsden
Build fixes for the benchmark
b5d7d86
@ilumsden
Optimization 1: adds feature to DTL to allocate buffers for networkin…
22a80d7 
…g/data transfer
@ilumsden ilumsden force-pushed the ucx_opts branch 2 times, most recently from f1ed369 to 39570ce Compare January 10, 2024 22:38
@JaeseungYeom JaeseungYeom mentioned this pull request Jan 19, 2024
Closed
@JaeseungYeom JaeseungYeom mentioned this pull request Jan 21, 2024
Closed
@ilumsden
Copy link
Collaborator Author

Verify that this was worked into another PR. If so, close.

@ilumsden ilumsden closed this Jun 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@ilumsden ilumsden

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ilumsden @hariharan-devarajan