what(): parallel_for failed: cudaErrorNoKernelImageForDevice: no kernel image is available for execution on the device

[Light--]

bug描述 Describe the Bug

---

[2025-01-20 16:56:34,709] [ INFO] lr_scheduler total_step: 385556
W0120 16:56:34.715422 106570 gpu_resources.cc:106] The GPU compute capability in your current machine is 90, which is not supported by Paddle, it is recommended to install the corresponding wheel package according to the installation information on the official Paddle website.
W0120 16:56:34.715462 106570 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 9.0, Driver API Version: 12.2, Runtime API Version: 11.2
W0120 16:56:34.715466 106570 gpu_resources.cc:164] device: 0, cuDNN Version: 8.1.
terminate called after throwing an instance of 'thrust::system::system_error'
what(): parallel_for failed: cudaErrorNoKernelImageForDevice: no kernel image is available for execution on the device

---

C++ Traceback (most recent call last):

0 paddle::pybind::ThrowExceptionToPython(std::__exception_ptr::exception_ptr)

---

Error Message Summary:

FatalError: Process abort signal is detected by the operating system.
[TimeInfo: *** Aborted at 1737363394 (unix time) try "date -d @1737363394" if you are using GNU date ***]
[SignalInfo: *** SIGABRT (@0x1a04a) received by PID 106570 (TID 0x7f02b5409b80) from PID 106570 ***]

LAUNCH INFO 2025-01-20 16:58:26,412 Exit code -6

• bash cluster/train_paddle.sh
  start paddle training ...

其他补充信息 Additional Supplementary Information

No response

[bukejiyu]

hello,you can try using this package, which has been compiled with support for compute capability 9.0.
python -m pip install paddlepaddle-gpu==3.0.0rc0 -i https://www.paddlepaddle.org.cn/packages/stable/cu123/
https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html