simulate_replay fails to link related to dumpStack

[nlslatt]

Describe the bug
tools/workload_replay/simulate_replay on Gitlab CI using clang 16.0.6:

ld.lld: error: undefined symbol: _Ux86_64_getcontext
ld.lld: error: undefined symbol: _ULx86_64_init_local
ld.lld: error: undefined symbol: _ULx86_64_get_reg
ld.lld: error: undefined symbol: _ULx86_64_get_proc_name
ld.lld: error: undefined symbol: _ULx86_64_step

all with the following note:

>>> referenced by stack_out.cc
>>>               stack_out.cc.o:(vt::debug::stack::dumpStack[abi:cxx11](int)) in archive src/libvt-release.a

[cz4rs]

@nlslatt
I was unable to reproduce this locally. Can you post CMake configuration log and linker command used?

Here's what I have locally from ninja -v simulate_replay:

/usr/bin/clang++ -g -rdynamic -Wl,-rpath -Wl,/usr/lib64/mpich/lib -Wl,--enable-new-dtags tools/workload_replay/CMakeFiles/simulate_replay.dir/simulate_replay.cc.o -o tools/workload_replay/simulate_replay  -Wl,-rpath,/usr/lib64/mpich/lib  lib/brotli/libbrotlicommon-static.a  lib/brotli/libbrotlidec-static.a  lib/brotli/libbrotlienc-static.a  lib/libfort/lib/libfort.a  -lunwind  src/libvt-debug.a  -ldl  /usr/lib64/libz.so  lib/fmt/libfmt-vtd.a  lib/EngFormat-Cpp/libEngFormat-Cpp.a  lib/checkpoint/src/libmagistrate.a  lib/brotli/libbrotlidec-static.a  lib/brotli/libbrotlienc-static.a  lib/brotli/libbrotlicommon-static.a  -lm  lib/libfort/lib/libfort.a  -lunwind  -ldl  /usr/lib64/mpich/lib/libmpicxx.so  /usr/lib64/mpich/lib/libmpi.so  /usr/lib64/libz.so

[cz4rs]

Misc notes:

• some libunwind docs about multiarchitecture support
• general docs confirm that we use UNW_LOCAL_ONLY correctly
• we use CMake for libunwind header detection; I'm assuming this works fine because the error only shows up at the linking stage

[lifflander]

Configure Command: "/projects/aue/hpc/builds/x86_64/rhel8/f6a36cb8/tooling-sprint-24.02/install/linux-rhel8-x86_64/gcc-10.3.0/cmake-3.27.7-f3bfcza/bin/cmake" "-DCMAKE_EXPORT_COMPILE_COMMANDS=1" "-Dvt_test_trace_runtime_enabled=0" "-Dvt_lb_enabled=1" "-Dvt_trace_enabled=0" "-Dvt_trace_only=0" "-Dvt_doxygen_enabled=0" "-Dvt_mimalloc_enabled=0" "-Dvt_asan_enabled=0" "-Dvt_ubsan_enabled=0" "-Dvt_werror_enabled=0" "-Dvt_pool_enabled=1" "-Dvt_build_extended_tests=1" "-Dvt_zoltan_enabled=0" "-Dvt_production_build_enabled=0" "-Dvt_unity_build_enabled=0" "-Dvt_diagnostics_enabled=1" "-Dvt_diagnostics_runtime_enabled=0" "-Dvt_fcontext_enabled=0" "-Dvt_fcontext_build_tests_examples=0" "-Dvt_rdma_tests_enabled=0" "-Dvt_code_coverage=0" "-DMI_INTERPOSE:BOOL=ON" "-DMI_OVERRIDE:BOOL=ON" "-Dvt_mpi_guards=0" "-DMPI_EXTRA_FLAGS=" "-DCMAKE_BUILD_TYPE=Release" "-DMPI_C_COMPILER=/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/clang-16.0.6/openmpi-4.1.6-hfszinc/bin/mpicc" "-DMPI_CXX_COMPILER=/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/clang-16.0.6/openmpi-4.1.6-hfszinc/bin/mpic++" "-DCMAKE_CXX_COMPILER=/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/clang-16.0.6/install/linux-rhel8-x86_64/clang-16.0.6/llvm-16.0.6-dk6jfj6/bin/clang++" "-DCMAKE_C_COMPILER=/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/clang-16.0.6/install/linux-rhel8-x86_64/clang-16.0.6/llvm-16.0.6-dk6jfj6/bin/clang" "-DCMAKE_EXE_LINKER_FLAGS=" "-Dmagistrate_ROOT=/projects/darma/gitlab-ci/jliffla/vt_build_gcc12-develop/4DsNyyUC/002//checkpoint/install" "-DCMAKE_PREFIX_PATH=/projects/aue/hpc/builds/x86_64/rhel8/5f95e974/tpls-intel-2023.2.0/install/linux-rhel8-x86_64/oneapi-2023.2.0/intel-oneapi-mkl-2023.2.0-wyfvc7j:/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/tpls-gcc-10.3.0-openmpi-4.1.6/install/linux-rhel8-x86_64/gcc-10.3.0/boost-1.83.0-2zgc4lm:/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/tpls-gcc-10.3.0-openmpi-4.1.6/install/linux-rhel8-x86_64/gcc-10.3.0/netcdf-c-4.9.2-tusldg3:/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/tpls-gcc-10.3.0-openmpi-4.1.6/install/linux-rhel8-x86_64/gcc-10.3.0/parallel-netcdf-1.12.3-ulfp6ow:/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/tpls-gcc-10.3.0-openmpi-4.1.6/install/linux-rhel8-x86_64/gcc-10.3.0/hdf5-1.14.2-76haqne:/projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/clang-16.0.6/openmpi-4.1.6-hfszinc:/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/clang-16.0.6/install/linux-rhel8-x86_64/clang-16.0.6/llvm-16.0.6-dk6jfj6:/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/tooling/install/linux-rhel8-x86_64/gcc-10.3.0/ninja-1.11.1-cjf3fb5:/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/tooling/install/linux-rhel8-x86_64/gcc-10.3.0/git-lfs-3.3.0-d4wknp2:/projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/tooling/install/linux-rhel8-x86_64/gcc-10.3.0/git-2.40.0-b7pxk5e:/projects/aue/hpc/builds/x86_64/rhel8/f6a36cb8/tooling-sprint-24.02/install/linux-rhel8-x86_64/gcc-10.3.0/cmake-3.27.7-f3bfcza:/projects/aue/hpc/builds/x86_64/rhel8/3aa2f152/tooling-24.08/install/linux-rhel8-x86_64/gcc-10.3.0/binutils-2.41-7d4dca2:/projects/aue/hpc/builds/x86_64/rhel8/3aa2f152/tooling-24.08/install/linux-rhel8-x86_64/gcc-10.3.0/zlib-1.3-3ighou7" "-DCMAKE_INSTALL_PREFIX=/projects/darma/gitlab-ci/jliffla/vt_build_gcc12-develop/4DsNyyUC/002//vt/install" "-Dvt_ci_build=0" "-Dvt_debug_verbose=" "-Dvt_tests_num_nodes=" "-Dvt_no_color_enabled=0" "-DBUILD_SHARED_LIBS=0" "-DCMAKE_JOB_POOLS=default_pool=8" "-DCMAKE_JOB_POOL_COMPILE='default_pool'" "-DCMAKE_JOB_POOL_LINK='default_pool'" "-GNinja" "-S/ascldap/users/jliffla/.jacamar-ci/builds/4DsNyyUC/002/darma-tasking/vt" "-B/projects/darma/gitlab-ci/jliffla/vt_build_gcc12-develop/4DsNyyUC/002/vt" 

-- The C compiler identification is Clang 16.0.6
-- The CXX compiler identification is Clang 16.0.6
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/clang-16.0.6/install/linux-rhel8-x86_64/clang-16.0.6/llvm-16.0.6-dk6jfj6/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/clang-16.0.6/install/linux-rhel8-x86_64/clang-16.0.6/llvm-16.0.6-dk6jfj6/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for include file malloc.h
-- Looking for include file malloc.h - found
-- Looking for include file malloc/malloc.h
-- Looking for include file malloc/malloc.h - not found
-- Looking for include file mach/mach.h
-- Looking for include file mach/mach.h - not found
-- Looking for include file sys/resource.h
-- Looking for include file sys/resource.h - found
-- Looking for include file unistd.h
-- Looking for include file unistd.h - found
-- Looking for include file inttypes.h
-- Looking for include file inttypes.h - found
-- Looking for include file libunwind.h
-- Looking for include file libunwind.h - found
-- Looking for include file execinfo.h
-- Looking for include file execinfo.h - found
-- Looking for mstats
-- Looking for mstats - not found
-- Looking for popen
-- Looking for popen - found
-- Looking for pclose
-- Looking for pclose - found
-- Looking for sbrk
-- Looking for sbrk - found
-- Looking for getpid
-- Looking for getpid - found
-- Looking for sysconf
-- Looking for sysconf - found
-- Looking for mallinfo
-- Looking for mallinfo - found
-- Looking for mallinfo2
-- Looking for mallinfo2 - not found
-- Looking for getrusage
-- Looking for getrusage - found
-- Looking for sysinfo
-- Looking for sysinfo - found
-- Looking for mach_task_self
-- Looking for mach_task_self - not found
-- VT: Found ccache binary: /usr/bin/ccache; adding launch rule
-- CMAKE_CXX_STANDARD: 17
-- Running tests and examples with additional MPI flags = 
-- find_package_local: pkg name="magistrate"
-- Found MPI_C: /projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/clang-16.0.6/openmpi-4.1.6-hfszinc/lib/libmpi.so (found version "3.1") 
-- Found MPI_CXX: /projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/clang-16.0.6/openmpi-4.1.6-hfszinc/lib/libmpi.so (found version "3.1") 
-- Found MPI: TRUE (found version "3.1")  
-- MPI max nproc: 48
-- MPI proc test list: 1;2;4;8;16;32
-- MPI exec: /projects/aue/hpc/deploy/b306e83c/linux-rhel8-x86_64/clang-16.0.6/openmpi-4.1.6-hfszinc/bin/mpiexec
-- Found ZLIB: /projects/aue/hpc/builds/x86_64/rhel8/3aa2f152/tooling-24.08/install/linux-rhel8-x86_64/gcc-10.3.0/zlib-1.3-3ighou7/lib/libz.so (found version "1.3")  
-- Found Perl: /usr/bin/perl (found version "5.26.3") 
-- Building VT with load balancing enabled
-- Building VT with trace disabled
-- Building VT with additional target for VT in trace-only mode disabled
-- Building VT with priority bits per level: 3 disabled
-- Building VT with runtime tracing (for testing) disabled
-- Building VT with memory pool enabled
-- Building VT with bit check overflow disabled
-- Building VT with fcontext (ULT) disabled
-- Building VT with mimalloc disabled
-- Building VT with user MPI prevention guards disabled
-- Building VT with Zoltan disabled
-- Building VT with verbose debug printing disabled
-- Building VT with RDMA tests disabled
-- Building VT with runtime performance metrics/stats disabled
-- Building VT with diagnostics (performance stats) enabled
-- Building VT with libfort enabled
-- Building VT with production build disabled
-- Building VT with CI build disabled
-- Building VT with --vt_no_color set to true by default disabled
-- Building VT with code coverage disabled
-- Building VT with `gold' linker enabled
-- Building VT with unity build disabled
-- Building VT with mimalloc disabled
-- Building VT with mimalloc using static linking enabled
-- Building VT with address sanitizer disabled
-- Building VT with undefined behavior sanitizer disabled
-- Building VT with -Werror disabled
-- Building VT with tests enabled
-- Building VT with tools enabled
-- Building VT with examples enabled
-- Building VT with generate lb files disabled
-- Building VT with Building vt with included fmt library
-- Weakened message contract checking. Trivially copyable checks are disabled as a type with a const member not detected as trivially copyable. Compile with another compiler (such as Clang) for increased static validation.
-- Weakened message contract checking. Messages cannot be detected for correct presense (or absense) of a 'serialize' method. Compile with another compiler (such as Clang) for increased static validation.
CMake Deprecation Warning at lib/libfort/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- libfort build options:  wchar support = ON;  build type = common
CMake Deprecation Warning at lib/yaml-cpp/CMakeLists.txt:2 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Deprecation Warning at lib/EngFormat-Cpp/CMakeLists.txt:2 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Deprecation Warning at lib/json/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Using the multi-header code from /ascldap/users/jliffla/.jacamar-ci/builds/4DsNyyUC/002/darma-tasking/vt/lib/json/include/
-- Build type is 'Release'
-- Performing Test BROTLI_EMSCRIPTEN
-- Performing Test BROTLI_EMSCRIPTEN - Failed
-- Compiler is not EMSCRIPTEN
-- Building brotli in portable mode
-- Looking for log2
-- Looking for log2 - not found
-- Looking for log2
-- Looking for log2 - found
-- Found Git: /projects/aue/hpc/builds/x86_64/rhel8/ba17d7f2/tooling/install/linux-rhel8-x86_64/gcc-10.3.0/git-2.40.0-b7pxk5e/bin/git (found version "2.40.0") 
-- Git DIR: /ascldap/users/jliffla/.jacamar-ci/builds/4DsNyyUC/002/darma-tasking/vt/.git
-- Git HEAD file: "/ascldap/users/jliffla/.jacamar-ci/builds/4DsNyyUC/002/darma-tasking/vt/.git/HEAD"
-- Performing Test flag_supported
-- Performing Test flag_supported - Success
-- Performing Test flag_supported
-- Performing Test flag_supported - Success
-- Performing Test flag_supported
-- Performing Test flag_supported - Success
-- Performing Test flag_supported
-- Performing Test flag_supported - Success
-- Performing Test flag_supported
-- Performing Test flag_supported - Success
-- Performing Test flag_supported
-- Performing Test flag_supported - Success
-- Performing Test flag_supported
-- Performing Test flag_supported - Success
-- Performing Test flag_supported
-- Performing Test flag_supported - Success
-- Performing Test flag_supported
-- Performing Test flag_supported - Success
-- Performing Test LINKER_SUPPORTS_RDYNAMIC
-- Performing Test LINKER_SUPPORTS_RDYNAMIC - Success
-- VT: building tools
-- VT: building examples
-- VT: building tests
-- Building VT with extended testing
-- Found Python: /usr/bin/python3.9 (found version "3.9.16") found components: Interpreter 
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Test file /ascldap/users/jliffla/.jacamar-ci/builds/4DsNyyUC/002/darma-tasking/vt/tests/perf/comm_cost_curve.cc is present on EXCLUDED_PERF_TEST_FILES list. Skipping!
-- Test file /ascldap/users/jliffla/.jacamar-ci/builds/4DsNyyUC/002/darma-tasking/vt/tests/perf/memory_checker.cc is present on EXCLUDED_PERF_TEST_FILES list. Skipping!
-- Configuring done (23.8s)
-- Generating done (3.6s)
CMake Warning:
  Manually-specified variables were not used by the project:

    MI_INTERPOSE
    MI_OVERRIDE
    vt_fcontext_build_tests_examples


-- Build files have been written to: /projects/darma/gitlab-ci/jliffla/vt_build_gcc12-develop/4DsNyyUC/002/vt