fix broken cpp integration caused by #567

[tsu-bin]

Hi, cpp integration was broken again by #567, please be aware that there are cpp test, cpp benchmark and also tvm integration, they all relay on cmake build.

[zhyncs]

src/tvm_wrapper.cu(690): error: identifier "BatchQKApplyRotaryInPlace" is undefined
    if (q->dtype.code == kDLFloat && q->dtype.bits == 16) { using dtype = half; {if (indptr->dtype.code == kDLInt && indptr->dtype.bits == 32) { using idtype = int32_t; { cudaError_t status = BatchQKApplyRotaryInPlace( static_cast<dtype*>(q->data), static_cast<dtype*>(k->data), static_cast<idtype*>(indptr->data), static_cast<idtype*>(offsets->data), batch_size, num_qo_heads, num_kv_heads, head_dim, q_stride_n, q_stride_h, k_stride_n, k_stride_h, false, rope_scale, rope_theta); if (status != cudaSuccess) { ::tvm::runtime::detail::LogFatal("/workspace/persistent-storage/flashinfer_dev/src/tvm_wrapper.cu", 698).stream() << "FlashInfer CUDA kernel error " << cudaGetErrorString(status); } } } else { ::tvm::runtime::detail::LogFatal("/workspace/persistent-storage/flashinfer_dev/src/tvm_wrapper.cu", 691).stream() << "Unsupported data type " << indptr->dtype.code; }} } else { ::tvm::runtime::detail::LogFatal("/workspace/persistent-storage/flashinfer_dev/src/tvm_wrapper.cu", 690).stream() << "Unsupported data type " << q->dtype.code; }

@tsu-bin I verified your branch, and overall there are no major issues. This error wasn't introduced by you. Would it be convenient for you to fix it in the next PR? I'll merge this PR first. cc @yzh119

[tsu-bin]

hi @zhyncs I will look into this issue later, I think this is less urgent, since tvm / mlc only rely on specific commit point.
Would you help to merge this pr, then I can continue to rebase my current work.

[abcdabcd987]

oops. Sorry for breaking cpp

[yzh119]

Sorry for the late reply, yes this PR fixes #571, thank you!

[tsu-bin]

src/tvm_wrapper.cu(690): error: identifier "BatchQKApplyRotaryInPlace" is undefined
    if (q->dtype.code == kDLFloat && q->dtype.bits == 16) { using dtype = half; {if (indptr->dtype.code == kDLInt && indptr->dtype.bits == 32) { using idtype = int32_t; { cudaError_t status = BatchQKApplyRotaryInPlace( static_cast<dtype*>(q->data), static_cast<dtype*>(k->data), static_cast<idtype*>(indptr->data), static_cast<idtype*>(offsets->data), batch_size, num_qo_heads, num_kv_heads, head_dim, q_stride_n, q_stride_h, k_stride_n, k_stride_h, false, rope_scale, rope_theta); if (status != cudaSuccess) { ::tvm::runtime::detail::LogFatal("/workspace/persistent-storage/flashinfer_dev/src/tvm_wrapper.cu", 698).stream() << "FlashInfer CUDA kernel error " << cudaGetErrorString(status); } } } else { ::tvm::runtime::detail::LogFatal("/workspace/persistent-storage/flashinfer_dev/src/tvm_wrapper.cu", 691).stream() << "Unsupported data type " << indptr->dtype.code; }} } else { ::tvm::runtime::detail::LogFatal("/workspace/persistent-storage/flashinfer_dev/src/tvm_wrapper.cu", 690).stream() << "Unsupported data type " << q->dtype.code; }

@tsu-bin I verified your branch, and overall there are no major issues. This error wasn't introduced by you. Would it be convenient for you to fix it in the next PR? I'll merge this PR first. cc @yzh119

Hi @zhyncs just fix it by #582