Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Distconv channelwise softmax #2148

Open
wants to merge 23 commits into
base: develop
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
d41bf54
Adding channelwise softmax distconv unit test.
szaman19 Oct 26, 2022
b3cf148
Set up boilerplate
szaman19 Oct 26, 2022
d108b2f
Update cmake-ary
szaman19 Oct 26, 2022
0d95e9a
- Fixed old file naming issues
szaman19 Jan 26, 2023
5fc580c
Adding forward impl
szaman19 Jan 26, 2023
3368e4f
Moved shareed kernels to channelwise_softmax_kernels.cuh
szaman19 Jan 26, 2023
569c830
Compiling and linking correctly
szaman19 Jan 26, 2023
3a43037
Updated ci test to test split case
szaman19 Jan 28, 2023
52399f1
Adding some debug code to see why output is always
szaman19 Jan 28, 2023
cfd768d
Passing forward pass on CI
szaman19 Jan 30, 2023
ce7896f
Passing CI tests
szaman19 Jan 30, 2023
af77dcc
- Added model compile-time checks on the shape of the input when dist…
szaman19 Jan 30, 2023
a2b1760
Strange behavior on CI. Every couple of gradient checks fail...
szaman19 Jan 30, 2023
281aafd
Passing CI tests
szaman19 Jan 30, 2023
4930744
Updated implementation to incorporate updated channelwise softmax API
szaman19 Jun 10, 2024
4b57509
Added guard on double ETI
szaman19 Jun 10, 2024
4896b5d
Updated CI test with new environment imports
szaman19 Jun 10, 2024
66f5ee3
Update ci_test/unit_tests/test_unit_layer_channelwise_softmax_distcon…
szaman19 Jun 11, 2024
8eef5a0
Updated year on textsd
szaman19 Jun 18, 2024
e9fcd84
Updated instantiation code
szaman19 Jun 18, 2024
602eeab
Updated copyright years
szaman19 Jun 18, 2024
c888476
Remove comment after applying PR suggestions
szaman19 Jun 25, 2024
bb876fa
Fix a couple issues with the auto-detection of the NVCC_GENCODE in NC…
benson31 Jun 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Set up boilerplate
szaman19 committed Jun 17, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit b3cf148c3ac69a1121a45e3acd451ce864e29e8f
3 changes: 3 additions & 0 deletions include/lbann/layers/misc/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -40,5 +40,8 @@ set_full_path(THIS_DIR_HEADERS
variance.hpp
)

if (LBANN_HAS_DISTCONV)
add_subdirectory(distconv)
endif()
# Propagate the files up the tree
set(HEADERS "${HEADERS}" "${THIS_DIR_HEADERS}" PARENT_SCOPE)
Empty file.
Empty file.
10 changes: 10 additions & 0 deletions include/lbann/utils/distconv.hpp
Original file line number Diff line number Diff line change
@@ -53,6 +53,16 @@
#include "p2p/p2p.hpp"
#endif // DISTCONV_HAS_P2P

#include "lbann/layers/learning/distconv/distconv_layers.hpp"
#include "lbann/layers/math/distconv/distconv_matmul.hpp"

#ifdef LBANN_HAS_NVSHMEM
#include "lbann/layers/transform/distconv/distconv_scatter.hpp"
#include "lbann/layers/transform/distconv/distconv_gather.hpp"
#include "lbann/layers/transform/distconv/distconv_nvshmem_vector_addressing.hpp"
#endif // LBANN_HAS_NVSHMEM

#include "lbann/layers/misc/distconv/distconv_channelwise_softmax.hpp"
namespace lbann {

inline auto default_hydrogen_stream()
4 changes: 4 additions & 0 deletions src/layers/misc/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -66,6 +66,10 @@ endif ()
# Add the subdirectories
add_subdirectory(cereal_registration)

if (LBANN_HAS_DISTCONV)
add_subdirectory(distconv)
endif()

# Propagate the files up the tree
set(SOURCES "${SOURCES}" "${THIS_DIR_SOURCES}" PARENT_SCOPE)
set(GPU_SOURCES "${GPU_SOURCES}" "${THIS_DIR_CU_SOURCES}" PARENT_SCOPE)
Empty file.
Empty file.