You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, ollama 7b/70b models are running comparatively slow on Intel systems(CPU). Also, it has support for AMD GPUs and Nvidia but not Intel. The 70b/7b llama hf model that I used from https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/254-llm-chatbot/254-llm-chatbot.ipynb is faster than from ollama. I feel OpenVINO acceleration gives better boost here. I am sorry but I am unable to get the latest update on support of these models on openvino model_server.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Currently, ollama 7b/70b models are running comparatively slow on Intel systems(CPU). Also, it has support for AMD GPUs and Nvidia but not Intel. The 70b/7b llama hf model that I used from https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/254-llm-chatbot/254-llm-chatbot.ipynb is faster than from ollama. I feel OpenVINO acceleration gives better boost here. I am sorry but I am unable to get the latest update on support of these models on openvino model_server.
Beta Was this translation helpful? Give feedback.
All reactions