Feature Request: add a new repo for convertion of gguf

[gogo2464]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Hello.

I would like to split convert_hf_to_gguf.py to a new repo. Can you create another package just for conversion.

Motivation

I am always frustruated when I want to convert gguf files because I have to run:

git clone https://github.com/ggerganov/llama.cpp.git
pip install -r llama.cpp/requirements.txt
python llama.cpp/convert_hf_to_gguf.py ./OUT-MODEL/ --outfile model.gguf --outtype q8_0

While I just want to convert. I do not need the llamacpp inference model.

Possible Implementation

Python to a pypi package. Or if possible in C/C++/rust for performance. I personally love rust.

[taronaeo]

While I just want to convert. I do not need the llamacpp inference model.

When you run pip install -r llama.cpp/requirements.txt, you are not actually compiling the inference engine. That would require cmake and your system's compilation flags to do so. The steps you have listed above is essentially just the gguf-py package where it contains the GGUFReader and GGUFWriter to support convert_hf_to_gguf.py's operations.

[gogo2464]

Yes. I just expect the pure convert_hf_to_gguf.py operation.