AWS Instances that best fit BitNet. Config of demo presented #263
Unanswered
niranjanakella
asked this question in
Q&A
Replies: 1 comment
-
And @tsong-ms, why are the outputs different on the demo CPU inference compared to what I am running locally. Can you please share the inference config of the demo. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hey, I really appreciate the work. I would like to know the AWS instance types with just CPU that best fit the inference through BitNet. I tried with t4g.xlarge, but the inference speeds are pretty low (maybe due to ARM). I haven't tried an x86 CPU yet.
Would be awesome to see a short set of architecture configs that would work well with Bitnet inference.
I am more interested to know what CPU arch/type was used for the demo presented on the repo.
Thank you for the cool work.
Beta Was this translation helpful? Give feedback.
All reactions