Awq model export for cpu is not supported in wins

[belovedxixi]

Describe the bug
Awq model export for cpu is not supported

To Reproduce
python3 builder.py -i awq_model_dir -o output_folder -p int4 -e cpu -c cache_dir

Screenshots

Additional context
It seems like hf 's file can not support awq_config for cpu

[kunal-vaishnavi]

According to Hugging Face, this error is raised when torch.cuda.is_available() = False. When torch.cuda.is_available() = False, this means that torch cannot find CUDA installed on your machine.

As you mentioned, it appears that Hugging Face requires that the machine that loads an AwqConfig must have CUDA installed. Here are a couple of workaround options.

1. If your machine supports installing CUDA, verify that CUDA is installed and that torch.cuda.is_available() = True. Then run your command again.
2. If your machine doesn't support installing CUDA, modify transformers so that post_init() is not called to avoid raising the error. Here are some steps to do this.

# 1) Uninstall existing `transformers` package
$ pip uninstall -y transformers

# 2) Clone `transformers` repo
$ git clone https://github.com/huggingface/transformers

# 3) Navigate to the folder that `quantization_config.py` is in
$ cd transformers/src/transformers/utils/

# 3) Comment out `self.post_init()` in `quantization_config.py`. Here's the location:
# https://github.com/huggingface/transformers/blob/9d2ab8824c2a820a0ac9f2cd884ef838ca77b481/src/transformers/utils/quantization_config.py#L814
#
# This will prevent the post-init steps from running and this error should not get raised.

# 4) Install `transformers` from source
$ cd ../../../
$ pip install -e .

Then run your command again.

[natke]

@belovedxixi Did you try the above suggestions? Let us know how you went