remove ALPAKA_ASSERT_OFFLOAD, introduce ALPAKA_ASSERT_ACC

[psychocoderHPC]

fix #2186
fix #2001

Provide a device side assert ALPAKA_ASSERT_ACC that can be disabled by defining ALPAKA_DISABLE_ASSERT_ACC in the C++ code or by the CMake option alpaka_ASSERT_ACC_ENABLE

In #2186 the footnote says that assert on the device can have a huge impact on the performance, therefore I added the possibility to force disable the device side asserts.
The naming for the CMake option follows the name schema we use for other options in CMake.

[fwyzard]

Hi @psychocoderHPC, thanks for the PR.

Is it the intent that people should always use ALPAKA_ASSERT_ACC in device code ?

I ask because I'm wondering if the check on SYCL_EXT_ONEAPI_ASSERT should be moved into ALPAKA_ASSERT instead ?

Something like

/// The assert can be explicitly disabled by defining NDEBUG
#if defined(ALPAKA_ACC_SYCL_ENABLED) && defined(__SYCL_DEVICE_ONLY__)
// SYCL device compilation
#    if defined(SYCL_EXT_ONEAPI_ASSERT)
#        define ALPAKA_ASSERT(...) assert(__VA_ARGS__)
#    else
#        define ALPAKA_ASSERT(...) ALPAKA_NOOP(__VA_ARGS__)
#    endif
#else
// non-SYCL device compilation
#    define ALPAKA_ASSERT(...) assert(__VA_ARGS__)
#endif

/// Assert for usage within device code which can be disabled independently from NDEBUG.
#if defined(ALPAKA_DISABLE_ASSERT_ACC)
#    define ALPAKA_ASSERT_ACC(...) ALPAKA_NOOP(__VA_ARGS__)
#else
#    define ALPAKA_ASSERT_ACC(...) ALPAKA_ASSERT(__VA_ARGS__)
#endif

[fwyzard]

Another thing: when the "device" is actually the cpu (e.g. device code compiled for AccCpuSerial) I would prefer to keep the assert enabled.

So, if none of __CUDA_ARCH__, __HIP_DEVICE_COMPILE__ or __SYCL_DEVICE_ONLY__ are defined, ALPAKA_ASSERT_ACC should be equivalent to plain assert.

[psychocoderHPC]

Hi @psychocoderHPC, thanks for the PR.

Is it the intent that people should always use ALPAKA_ASSERT_ACC in device code ?

I ask because I'm wondering if the check on SYCL_EXT_ONEAPI_ASSERT should be moved into ALPAKA_ASSERT instead ?

Something like

/// The assert can be explicitly disabled by defining NDEBUG
#if defined(ALPAKA_ACC_SYCL_ENABLED) && defined(__SYCL_DEVICE_ONLY__)
// SYCL device compilation
#    if defined(SYCL_EXT_ONEAPI_ASSERT)
#        define ALPAKA_ASSERT(...) assert(__VA_ARGS__)
#    else
#        define ALPAKA_ASSERT(...) ALPAKA_NOOP(__VA_ARGS__)
#    endif
#else
// non-SYCL device compilation
#    define ALPAKA_ASSERT(...) assert(__VA_ARGS__)
#endif

/// Assert for usage within device code which can be disabled independently from NDEBUG.
#if defined(ALPAKA_DISABLE_ASSERT_ACC)
#    define ALPAKA_ASSERT_ACC(...) ALPAKA_NOOP(__VA_ARGS__)
#else
#    define ALPAKA_ASSERT_ACC(...) ALPAKA_ASSERT(__VA_ARGS__)
#endif

Yes from my side it was intended that we have an assert for the host site and the usage within the device code.
To control the assert behaviour typically NDEBUG is used. I do not want to change these default behavior for host code.
My intention was that we only to a non-standard way for device-side asserts.

Never the less I can change it if needed.

[psychocoderHPC]

Another thing: when the "device" is actually the cpu (e.g. device code compiled for AccCpuSerial) I would prefer to keep the assert enabled.

So, if none of __CUDA_ARCH__, __HIP_DEVICE_COMPILE__ or __SYCL_DEVICE_ONLY__ are defined, ALPAKA_ASSERT_ACC should be equivalent to plain assert.

We can handle asserts differently depending on which accelerator is enabled. The drawback would be that in this case, a kernel with asserts executed with the CUDA backend will behave slightly differently compared to the kernel that is running with a CPU accelerator.
If you use SYCL and compile for CPU than it is hard to know for the user if asserts are enabled or disabled.

[fwyzard]

If you use SYCL and compile for CPU than it is hard to know for the user if asserts are enabled or disabled.

Indeed, by "CPU" I really meant "compiled by the standard compiler, like gcc".

[fwyzard]

Never the less I can change it if needed.

I would say that if we document something like

use

• ALPAKA_ASSERT for a check that can can be used anywhere, and is controlled by NDEBUG
• ALPAKA_ASSERT_ACC for a check that can can be used anywhere, but is disabled by default on backends where it may have a high cost

then it should be changed.

If instead we document

use

• ALPAKA_ASSERT for a check that can can be used on the host, and is controlled by NDEBUG
• ALPAKA_ASSERT_ACC for a check that can can be used on the device, and is disabled by default on backends where it may have a high cost

there is no need to change it.

[fwyzard]

With some delay, here is what I have in mind:

// ALPAKA_ASSERT_ACC_IMPL is an assert-like macro.
// It can be disabled setting the ALPAKA_DISABLE_ASSERT_ACC preprocessor symbol or the NDEBUG preprocessor symbol.
#if !defined(ALPAKA_DISABLE_ASSERT_ACC)
#    define ALPAKA_ASSERT_ACC_IMPL(...) ALPAKA_ASSERT(__VA_ARGS__)
#else
#    define ALPAKA_ASSERT_ACC_IMPL(...) ALPAKA_NOOP(__VA_ARGS__)
#endif

// ALPAKA_ASSERT_ACC is an assert-like macro.
// In device code for a GPU or SYCL backend it can be disabled setting the ALPAKA_DISABLE_ASSERT_ACC preprocessor
// symbol or the NDEBUG preprocessor symbol. In device code for a native C++ CPU backend and in host code, it is
// equivalent to ALPAKA_ASSERT, and can be disabled setting the NDEBUG preprocessor symbol.
#if defined(ALPAKA_ACC_GPU_CUDA_ENABLED) && defined(__CUDA_ARCH__)
// CUDA device code
#    define ALPAKA_ASSERT_ACC(...) ALPAKA_ASSERT_ACC_IMPL(__VA_ARGS__)
#elif defined(ALPAKA_ACC_GPU_HIP_ENABLED) && defined(__HIP_DEVICE_COMPILE__)
// HIP/ROCm device code
#    define ALPAKA_ASSERT_ACC(...) ALPAKA_ASSERT_ACC_IMPL(__VA_ARGS__)
#elif defined(ALPAKA_ACC_SYCL_ENABLED) && defined(__SYCL_DEVICE_ONLY__)
// SYCL/oneAPI device code
#    if defined(SYCL_EXT_ONEAPI_ASSERT)
#        define ALPAKA_ASSERT_ACC(...) ALPAKA_ASSERT_ACC_IMPL(__VA_ARGS__)
#    else
#        define ALPAKA_ASSERT_ACC(...) ALPAKA_NOOP(__VA_ARGS__)
#    endif
// add here any other #elif conditions for non-CPU backends
// ...
#else
// CPU backend, or host code
#    define ALPAKA_ASSERT_ACC(...) ALPAKA_ASSERT(__VA_ARGS__)
#endif

I've split off the check on ALPAKA_DISABLE_ASSERT_ACC into a separate macro ALPAKA_ASSERT_ACC_IMPL to make the code more readable - but that's supposed to be just an implementation detail, not something the user code would call.

[psychocoderHPC]

I updated the PR with the suggestion from @fwyzard and added him as a co-author.

[psychocoderHPC]

PR can be merged, the failing CI test is not required and is already fixed with #2208