Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make VAMP usable as a C++ header-only library #31

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

wbthomason
Copy link
Contributor

This PR extends the CMake configuration for VAMP to expose a header-only library of the core implementation, for use in downstream C++ projects.

I have not tested these changes at all yet; they are very likely incomplete and/or buggy.
One thing to consider is how much we want to restructure the project to provide more convenience aliases in the headers vs. in the current binding C++ files.

@claytonwramsey
Copy link
Member

I'm trying to move to a CPM based approach to using this as a library. The first issue (so far):

Including vamp as a package like so:

file(
  DOWNLOAD
  https://github.com/cpm-cmake/CPM.cmake/releases/download/v0.40.1/CPM.cmake
  ${CMAKE_CURRENT_BINARY_DIR}/cmake/CPM.cmake
  EXPECTED_HASH SHA256=117cbf2711572f113bab262933eb5187b08cfc06dce0714a1ee94f2183ddc3ec
)
set(CPM_USE_LOCAL_PACKAGES ON)
include(${CMAKE_CURRENT_BINARY_DIR}/cmake/CPM.cmake)

CPMAddPackage("gh:KavrakiLab/vamp#8105e16e070db72029ee1a658ac793b25bf08742")
mkdir -p build/release
cd build/release
cmake ../..

I get the following error:

CMake Error at /usr/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python (missing: Python_INCLUDE_DIRS Development.Module)
  (found suitable version "3.11.2", minimum required is "3.8")
Call Stack (most recent call first):
  /usr/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.25/Modules/FindPython/Support.cmake:3240 (find_package_handle_standard_args)
  /usr/share/cmake-3.25/Modules/FindPython.cmake:519 (include)
  build/release/_deps/vamp-src/cmake/Dependencies.cmake:2 (find_package)
  build/release/_deps/vamp-src/CMakeLists.txt:46 (include)

I have two guesses as to fixes:

  • is CMake somehow reading semver strings wrong? 3.11.2 seems compatible to me.
  • Can we prevent including a Python dependency when building as a C++-only library? My guess is that we need to gate based on VAMP_BUILD_PYTHON_BINDINGS in Dependencies.cmake.

@wbthomason
Copy link
Contributor Author

I think the version issue is actually just that your system doesn't have the Python development headers installed. It says 3.11.2 is suitable, but that it couldn't find Development.Module. Pretty sure the other problem is that I forgot to push my changes to Dependencies.cmake; will fix later today.

@claytonwramsey
Copy link
Member

Seems like no dice -

-- The CXX compiler identification is GNU 12.2.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CPM: Adding package vamp@0 (b56f0a93d86fd6874bf722c3378cd9b9946a3276)
-- The C compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- CPM: vamp: Adding package nigh@0 (97130999440647c204e0265d05a997dbd8da4e70)
-- CPM: vamp: Adding package pdqsort@0 (b1ef26a55cdb60d236a5cb199c4234c704f46726)
-- CPM: vamp: Adding package SIMDxorshift@0 (857c1a01df53cf1ee1ae8db3238f0ef42ef8e490)
-- Configuring done
CMake Error in build/release/_deps/vamp-src/CMakeLists.txt:
  Target "vamp" INTERFACE_INCLUDE_DIRECTORIES property contains path:

    "/home/cwr3/projects/navvy/build/release/_deps/nigh-src/src"

  which is prefixed in the build directory.


CMake Error in build/release/_deps/vamp-src/CMakeLists.txt:
  Target "vamp" INTERFACE_INCLUDE_DIRECTORIES property contains path:

    "/home/cwr3/projects/navvy/build/release/_deps/nigh-src/src"

  which is prefixed in the build directory.Target "vamp"
  INTERFACE_INCLUDE_DIRECTORIES property contains path:

    "/home/cwr3/projects/navvy/build/release/_deps/nigh-src/src"

  which is prefixed in the source directory.


CMake Error in build/release/_deps/vamp-src/CMakeLists.txt:
  Target "vamp" INTERFACE_INCLUDE_DIRECTORIES property contains path:

    "/home/cwr3/projects/navvy/build/release/_deps/pdqsort-src"

  which is prefixed in the build directory.


CMake Error in build/release/_deps/vamp-src/CMakeLists.txt:
  Target "vamp" INTERFACE_INCLUDE_DIRECTORIES property contains path:

    "/home/cwr3/projects/navvy/build/release/_deps/pdqsort-src"

  which is prefixed in the build directory.Target "vamp"
  INTERFACE_INCLUDE_DIRECTORIES property contains path:

    "/home/cwr3/projects/navvy/build/release/_deps/pdqsort-src"

  which is prefixed in the source directory.


-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

I've checked build/release and it looks like the correct dependencies are being downloaded, so I'm not quite sure what to make of this.

@claytonwramsey
Copy link
Member

Looked around a little - maybe these two are related?

My guess is that something is declared as PUBLIC when it shouldn't be, but I can't say I fully understand how this works or why.

@wbthomason
Copy link
Contributor Author

I think this is a problem with the fact that we use CPM for our dependencies...
@zkingston We got hit with it sooner than expected, looks like.

Might be able to fix by marking some includes as private; I'll experiment.

@claytonwramsey
Copy link
Member

here's a link to a reproducing project:

https://github.com/claytonwramsey/vampdep

@claytonwramsey
Copy link
Member

claytonwramsey commented Oct 17, 2024

I manually rewrote my CMakeLists file to include all the transitive dependencies, which is a bit of a hack but it works for now. The latest version is here. I'm running up against a fun little compilation disaster right now:

-- CPM: Adding package vamp@0 (bd5d218684c1608a61306537e4532932364e3411)
-- CPM: vamp: Adding package nigh@0 (97130999440647c204e0265d05a997dbd8da4e70)
-- CPM: vamp: Adding package pdqsort@0 (b1ef26a55cdb60d236a5cb199c4234c704f46726)
-- CPM: vamp: Adding package SIMDxorshift@0 (857c1a01df53cf1ee1ae8db3238f0ef42ef8e490)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/cwr3/projects/vampdep/build/release
[ 50%] Building CXX object CMakeFiles/test.dir/src/test.cpp.o
In file included from /home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/vector/interface.hh:14,
                 from /home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/vector.hh:4,
                 from /home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/collision/shapes.hh:6,
                 from /home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/collision/environment.hh:5,
                 from /home/cwr3/projects/vampdep/src/test.cpp:2:
/home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/vector/utils.hh: In function ‘constexpr auto vamp::{anonymous}::apply(DataT, DataT) [with auto fn = vamp::SIMDVector<__vector(8) float>::mul<0>; DataT = std::array<__vector(8) float, 1>]’:
/home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/vector/utils.hh:156:31: note: the ABI for passing parameters with 32-byte alignment has changed in GCC 4.6
  156 |         inline constexpr auto apply(DataT l_data, DataT r_data) noexcept
      |                               ^~~~~
/home/cwr3/projects/vampdep/src/test.cpp: In static member function ‘static constexpr vamp::SIMDVector<__vector(8) float>::VectorT vamp::SIMDVector<__vector(8) float>::shift_right(VectorT, unsigned int) [with unsigned int <anonymous> = 0]’:
/home/cwr3/projects/vampdep/src/test.cpp:31:1: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
   31 | }
      | ^
In file included from /usr/lib/gcc/x86_64-linux-gnu/12/include/immintrin.h:43,
                 from /home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/vector/avx.hh:9,
                 from /home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/vector.hh:7:
/usr/lib/gcc/x86_64-linux-gnu/12/include/avxintrin.h:1443:1: error: inlining failed in call to ‘always_inline’ ‘__m256 _mm256_castsi256_ps(__m256i)’: target specific option mismatch
 1443 | _mm256_castsi256_ps (__m256i __A)
      | ^~~~~~~~~~~~~~~~~~~
/home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/vector/avx.hh:198:39: note: called from here
  198 |             return _mm256_castsi256_ps(_mm256_srli_epi32(_mm256_castps_si256(v), i));
      |                    ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/12/include/immintrin.h:47:
/usr/lib/gcc/x86_64-linux-gnu/12/include/avx2intrin.h:773:1: error: inlining failed in call to ‘always_inline’ ‘__m256i _mm256_srli_epi32(__m256i, int)’: target specific option mismatch
  773 | _mm256_srli_epi32 (__m256i __A, int __B)
      | ^~~~~~~~~~~~~~~~~
/home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/vector/avx.hh:198:39: note: called from here
  198 |             return _mm256_castsi256_ps(_mm256_srli_epi32(_mm256_castps_si256(v), i));
      |                    ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-linux-gnu/12/include/avxintrin.h:1437:1: error: inlining failed in call to ‘always_inline’ ‘__m256i _mm256_castps_si256(__m256)’: target specific option mismatch
 1437 | _mm256_castps_si256(__m256 __A)
      | ^~~~~~~~~~~~~~~~~~~
/home/cwr3/projects/vampdep/build/release/_deps/vamp-src/src/impl/vamp/vector/avx.hh:198:39: note: called from here
  198 |             return _mm256_castsi256_ps(_mm256_srli_epi32(_mm256_castps_si256(v), i));
      |                    ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [CMakeFiles/test.dir/build.make:76: CMakeFiles/test.dir/src/test.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:99: CMakeFiles/test.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

@claytonwramsey
Copy link
Member

Figured out the issue. For some reason I also need to manually set VAMP_ARCH in my own CMakeLists.txt. I had to copy over the following block from the VAMP CMakeLists.txt:

if(CMAKE_SYSTEM_PROCESSOR STREQUAL "x86_64")
	# Need explicit AVX2 for some MacOS clang versions
	set(VAMP_ARCH "-march=native -mavx2")
elseif(CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64" OR CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
	# ARM platforms (aarch64 / arm64)
	set(VAMP_ARCH "-mcpu=native -mtune=native")
else()
	message(FATAL_ERROR "Unsupported architecture ${CMAKE_SYSTEM_PROCESSOR}")
endif()

The resulting CMakeLists for my project is a little ridiculous but it seems workable enough for now.

@pac48
Copy link

pac48 commented Nov 10, 2024

@claytonwramsey The reason VAMP_ARCH needs to be set in your test project is because the scope of that variable is not global. A simpler solution than copying the block from VAMP is to just include the compiler settings directly, e.g include(${vamp_SOURCE_DIR}/cmake/CompilerSettings.cmake). Another potential option may be to add the needed compiler options to the VAMP target directly as public.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants