[Int4-AWQ] Fix AWQ Marlin check for ROCm

[hegemanjw4amd]

This commit resolves an issue with the new AWQ Marlin support added in upstream. AWQ Marlin is not yet supported for ROCm in vllm, but vllm will override AWQ quantization with AWQ Marlin if quantization parameters are compatible without comprehensively checking for platform support. This commit fixes this problem in the case of ROCm by adding an is_hip() check.

[shajrawi]

Can we have that as an upstream PR?

[rasmith]

Hmm, maybe add in upstream. There is already some code to check is_hip if AWQ is being used and set the VLLM_USE_TRITON_AWQ if it is not already set. Currently, "awq_marlin" is used as the quantization method when an AWQ model is loaded. It would be nice if an is_hip check were done and the "awq" quantization method was selected instead if is_hip returned true.

[hegemanjw4amd]

Hmm, maybe add in upstream. There is already some code to check is_hip if AWQ is being used and set the VLLM_USE_TRITON_AWQ if it is not already set. Currently, "awq_marlin" is used as the quantization method when an AWQ model is loaded. It would be nice if an is_hip check were done and the "awq" quantization method was selected instead if is_hip returned true.

Okay, I have moved the check to a higher level to be more in line with the current rocm_supported_quantization list.

I would like to put this into ROCm/vllm first, because we need it for QA.

[rasmith]

nit: can the breaks on 292/296 be removed and just use a break on 297?

[hegemanjw4amd]

nit: can the breaks on 292/296 be removed and just use a break on 297?

That wouldn't quite be the same logically.