Error on Mac

[AndyBlocker]

/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/site-packages/transformers/generation/utils.py:725: UserWarning: MPS: no support for int64 repeats mask, casting it to int32 (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/mps/operations/Repeat.mm:236.) input_ids = input_ids.repeat_interleave(expand_size, dim=0) Exception in thread Thread-6: Traceback (most recent call last): File "/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/threading.py", line 980, in _bootstrap_inner self.run() File "/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/threading.py", line 917, in run self._target(*self._args, **self._kwargs) File "/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/site-packages/transformers/generation/utils.py", line 1588, in generate return self.sample( File "/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/site-packages/transformers/generation/utils.py", line 2642, in sample outputs = self( File "/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/Users/andyblocker/.cache/huggingface/modules/transformers_modules/ehartford/WizardLM-Uncensored-Falcon-7b/a95d8a001ec405c7d33baf704a190066949f2072/modelling_RW.py", line 753, in forward transformer_outputs = self.transformer( File "/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/Users/andyblocker/.cache/huggingface/modules/transformers_modules/ehartford/WizardLM-Uncensored-Falcon-7b/a95d8a001ec405c7d33baf704a190066949f2072/modelling_RW.py", line 590, in forward inputs_embeds = self.word_embeddings(input_ids) File "/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/site-packages/torch/nn/modules/sparse.py", line 162, in forward return F.embedding( File "/Users/andyblocker/mambaforge/envs/llm-as-chatbot/lib/python3.9/site-packages/torch/nn/functional.py", line 2210, in embedding return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse) TypeError: Trying to convert BFloat16 to the MPS backend but it does not have support for that dtype.

Error on MBP M1 Pro with 16G RAM, maybe from the newer version of torch? tried 2.0.0, 2.0.1, 1.13.1, all donnot work.

[deep-diver]

for some models which uses bfloat16 data type, it can't be loaded up on mac.