AWS Instances that best fit BitNet. Config of demo presented

[niranjanakella]

Hey, I really appreciate the work. I would like to know the AWS instance types with just CPU that best fit the inference through BitNet. I tried with t4g.xlarge, but the inference speeds are pretty low (maybe due to ARM). I haven't tried an x86 CPU yet.

Would be awesome to see a short set of architecture configs that would work well with Bitnet inference.

I am more interested to know what CPU arch/type was used for the demo presented on the repo.

Thank you for the cool work.

[niranjanakella]

And @tsong-ms, why are the outputs different on the demo CPU inference compared to what I am running locally. Can you please share the inference config of the demo.