Error of fused_dense module of flash_attn

[Jenny199099]

Hi author, I have compiled and installed fused_dense_lib successfully. But when I tried to run the finetuning code, I encountered this error: "RuntimeError: linear_act_forward failed." which is due to the line 291 in InternVideo2/single_modality/models/internvideo2.py. The complete error is as shown below:

Traceback (most recent call last):
File "", line 1, in
File "/nvme/miniconda3/envs/internvideo/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/nvme/miniconda3/envs/internvideo/lib/python3.8/site-packages/flash_attn/ops/fused_dense.py", line 457, in forward
out = fused_mlp_func(
File "/nvme/miniconda3/envs/internvideo/lib/python3.8/site-packages/flash_attn/ops/fused_dense.py", line 391, in fused_mlp_func
return FusedMLPFunc.apply(
File "/nvme/miniconda3/envs/internvideo/lib/python3.8/site-packages/torch/cuda/amp/autocast_mode.py", line 97, in decorate_fwd
return fwd(*args, **kwargs)
File "/nvme/miniconda3/envs/internvideo/lib/python3.8/site-packages/flash_attn/ops/fused_dense.py", line 257, in forward
output1, *rest = fused_dense_cuda.linear_act_forward(
RuntimeError: linear_act_forward failed.

Could you please help me to fix this? Thank you very much.

[Andy1621]

I have not met this problem. Maybe you can refer to this issue Dao-AILab/flash-attention#289 (comment).

[Jenny199099]

I have not met this problem. Maybe you can refer to this issue Dao-AILab/flash-attention#289 (comment).

Thank you for your reply. I follow the above comment to not use fused_mlp with DeepSpeed. This can fix this error.