[Bugfix] Fix quark fp8 format loading on AMD GPUs

[fxmarty-amd]

#10765 was merged into vllm to support Quark quantized models. Unfortunately, tests are somewhat limited and the issue fixed in this PR was not catched by the tests. Namely, if you compare

vllm/vllm/model_executor/layers/quantization/fp8.py

Lines 303 to 317 in 7a8987d

	if current_platform.is_rocm():
	weight, weight_scale, input_scale = \
	normalize_e4m3fn_to_e4m3fnuz(
	weight=weight,
	weight_scale=weight_scale,
	input_scale=layer.input_scale)
	if input_scale is not None:
	layer.input_scale = Parameter(input_scale,
	requires_grad=False)
	weight_scale, weight = requantize_with_max_scale(
	weight=weight,
	weight_scale=weight_scale,
	logical_widths=layer.logical_widths,
	)

and

vllm/vllm/model_executor/layers/quantization/quark/schemes/quark_w8a8_fp8.py

Lines 34 to 51 in 7a8987d

	if self.qscheme == "per_tensor":
	max_w_scale, weight = requantize_with_max_scale(
	weight=layer.weight,
	weight_scale=layer.weight_scale,
	logical_widths=layer.logical_widths,
	)
	if current_platform.is_rocm():
	weight, max_w_scale, input_scale = normalize_e4m3fn_to_e4m3fnuz(
	weight=weight,
	weight_scale=max_w_scale,
	input_scale=layer.input_scale)
	if input_scale is not None:
	layer.input_scale = Parameter(input_scale,
	requires_grad=False)
	layer.weight = Parameter(weight.t(), requires_grad=False)
	layer.weight_scale = Parameter(max_w_scale, requires_grad=False)

We see that the call order of normalize_e4m3fn_to_e4m3fnuz and requantize_with_max_scale is swapped for quark. This eventually loads in weights being different in case a checkpoint uses "quant_method": "quark" and is loaded in vllm.

We might want to add more tests or extend https://github.com/vllm-project/vllm/blob/main/tests/quantization/test_quark.py in this PR or in a later PR.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[BowenBao]

@fxmarty-amd let's update this PR with fix we discussed internally.

[kewang-xlnx]

ok

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @fxmarty-amd.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mgoin]

LGTM!