70B base model benchmarks? #87
BarfingLemurs
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I care about 70B 2.3-2.55bit comparisons, for use with a single 24gb. What are the default MMLU and perplexity scores when you use a 4bpw base llama model (or those model sizes most similar to GPTQ variants)?
P.S.
Is the codebase (the quantization conversion), in development?, the nature of converting these models require a lot of bandwidth and compute, so updating and sharing large models is more difficult.
Beta Was this translation helpful? Give feedback.
All reactions