Skip to content

Commit

Permalink
Add comment on how to download and covert to GGUF a T5 model
Browse files Browse the repository at this point in the history
  • Loading branch information
sogartar committed Nov 20, 2024
1 parent 78801cb commit 82bcdcf
Showing 1 changed file with 9 additions and 1 deletion.
10 changes: 9 additions & 1 deletion sharktank/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,13 +143,21 @@ def pytest_addoption(parser):
help="Llama3.1 405b fp8 model path",
)

# To obtain a T5 GGUF file you can use llama.cpp's convert_hf_to_gguf.py.
# https://github.com/ggerganov/llama.cpp/blob/9abe9eeae98b11fa93b82632b264126a010225ff/convert_hf_to_gguf.py
# E.g.
# git lfs install
# git clone https://huggingface.co/google/t5-v1_1-small
# convert_hf_to_gguf.py \
# --outfile t5-v1_1-small.gguf \
# --outtype=f32 \
# t5-v1_1-small
parser.addoption(
"--google-t5-v1-1-small-fp32-model-path",
type=Path,
default="/data/t5/small/google__t5-v1_1-small_fp32.gguf",
help="Google T5 v1.1 small fp32 model path",
)

parser.addoption(
"--google-t5-v1-1-xxl-fp32-model-path",
type=Path,
Expand Down

0 comments on commit 82bcdcf

Please sign in to comment.