Skip to content

Commit c07760b

Browse files
kchandankchandan
committed
Added ollama steps for windows
1 parent fe51af4 commit c07760b

File tree

1 file changed

+186
-0
lines changed

1 file changed

+186
-0
lines changed

‎projects/ollama-openwebui-chatbot/README.md

+186
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
2+
# Chatbot with Ollama
3+
4+
This repository contains a simple chatbot application built using OpenWebUI and an [Ollama](https://ollama.ai/) model backend for generating responses. The application takes user inputs via a text box and sends them to the Ollama backend to generate responses using a specified model.
5+
6+
## Features
7+
8+
- **Interactive Chatbot Interface**: Built with OpenWebUI to provide a simple UI for interactions.
9+
- **Ollama Backend Integration**: Connects to the Ollama backend server for LLM-based responses.
10+
- **Environment Configuration**: Easily configure the Ollama backend server IP using an environment variable.
11+
12+
13+
## Option 1: On your local Laptop ( Windows )
14+
15+
### Step 1: Ensure Docker Desktop and `nvidia-smi` are installed and Running
16+
17+
https://www.docker.com/products/docker-desktop/
18+
19+
```
20+
sudo docker ps
21+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
22+
```
23+
```
24+
>nvidia-smi.exe
25+
Sat Apr 19 13:35:25 2025
26+
+---------------------------------------------------------------------------------------+
27+
| NVIDIA-SMI 538.78 Driver Version: 538.78 CUDA Version: 12.2 |
28+
|-----------------------------------------+----------------------+----------------------+
29+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
30+
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
31+
| | | MIG M. |
32+
|=========================================+======================+======================|
33+
| 0 NVIDIA RTX A2000 8GB Lap... WDDM | 00000000:01:00.0 On | N/A |
34+
| N/A 44C P5 7W / 80W | 5570MiB / 8192MiB | 0% Default |
35+
| | | N/A |
36+
+-----------------------------------------+----------------------+----------------------+
37+
38+
+---------------------------------------------------------------------------------------+
39+
| Processes: |
40+
| GPU GI CI PID Type Process name GPU Memory |
41+
| ID ID Usage |
42+
|=======================================================================================|
43+
+---------------------------------------------------------------------------------------+
44+
```
45+
46+
### Step 2: Download Ollama with GPU support
47+
48+
```
49+
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
50+
```
51+
52+
Output
53+
54+
```
55+
PS C:\beCloudReady> docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
56+
Unable to find image 'ollama/ollama:latest' locally
57+
latest: Pulling from ollama/ollama
58+
f45c5ef3e181: Pull complete
59+
161508c220d5: Pull complete
60+
d9802f032d67: Pull complete
61+
0d15e460e575: Pull complete
62+
Digest: sha256:96b7667cb536ab69bfd5cc0c2bd1e29602218e076fe6d34f402b786f17b4fde0
63+
Status: Downloaded newer image for ollama/ollama:latest
64+
53ff742791ae2aa54b1af8fe16c09e91d730f4f8d753bf14e9a903cc6f13d991
65+
```
66+
67+
### Step 3: Download Llama 3.2 Model
68+
69+
```
70+
PS C:\beCloudReady> docker exec -it ollama ollama run llama3.2
71+
...
72+
>>> Hello how are you?
73+
I'm just a computer program, so I don't have feelings, but thank you for asking! How can I assist
74+
you today? Is there something on your mind that you'd like to chat about or ask for help with?
75+
I'm all ears (or rather, all text).
76+
77+
>>> Send a message (/? for help)
78+
```
79+
If your GPU has enough RAM, try deepseek 7B
80+
81+
```
82+
docker exec -it ollama ollama run deepseek-r1:7b
83+
```
84+
85+
### Step 3.1: Ensure Ollama is using GPU
86+
87+
```
88+
docker exec -it ollama ollama ps
89+
NAME ID SIZE PROCESSOR UNTIL
90+
llama3.2:latest a80c4f17acd5 4.0 GB 100% GPU 4 minutes from now
91+
```
92+
```
93+
docker exec -it ollama nvidia-smi
94+
Sat Apr 19 21:03:59 2025
95+
+---------------------------------------------------------------------------------------+
96+
| NVIDIA-SMI 535.183.04 Driver Version: 538.78 CUDA Version: 12.2 |
97+
|-----------------------------------------+----------------------+----------------------+
98+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
99+
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
100+
| | | MIG M. |
101+
|=========================================+======================+======================|
102+
| 0 NVIDIA RTX A2000 8GB Lap... On | 00000000:01:00.0 On | N/A |
103+
| N/A 43C P8 7W / 80W | 4172MiB / 8192MiB | 0% Default |
104+
| | | N/A |
105+
+-----------------------------------------+----------------------+----------------------+
106+
107+
+---------------------------------------------------------------------------------------+
108+
| Processes: |
109+
| GPU GI CI PID Type Process name GPU Memory |
110+
| ID ID Usage |
111+
|=======================================================================================|
112+
| 0 N/A N/A 209 C /ollama N/A |
113+
+---------------------------------------------------------------------------------------+
114+
```
115+
116+
### Step 3.2: Find the Bridge (internal IP) of the Ollama container
117+
118+
```
119+
docker inspect ollama | findstr "IP"
120+
```
121+
Expected Output
122+
123+
```
124+
docker inspect ollama | findstr "IP"
125+
"LinkLocalIPv6Address": "",
126+
"LinkLocalIPv6PrefixLen": 0,
127+
"SecondaryIPAddresses": null,
128+
"SecondaryIPv6Addresses": null,
129+
"GlobalIPv6Address": "",
130+
"GlobalIPv6PrefixLen": 0,
131+
"IPAddress": "172.17.0.2",
132+
"IPPrefixLen": 16,
133+
"IPv6Gateway": "",
134+
"IPAMConfig": null,
135+
"IPAddress": "172.17.0.2",
136+
"IPPrefixLen": 16,
137+
"IPv6Gateway": "",
138+
"GlobalIPv6Address": "",
139+
"GlobalIPv6PrefixLen": 0,
140+
```
141+
142+
In this example the IP of Ollama container is `172.17.0.2`
143+
144+
### Step 4: Run Open Web UI with Ollama container IP
145+
146+
```
147+
docker run -d -p 3000:8080 -e WEBUI_AUTH=False -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://"$env:OLLAMA_IP":11434 --name open-webui ghcr.io/open-webui/open-webui:main
148+
```
149+
150+
Check all Running containers
151+
```
152+
docker ps
153+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
154+
1500881ca4dc ghcr.io/open-webui/open-webui:main "bash start.sh" 3 minutes ago Up 3 minutes (healthy) 0.0.0.0:3000->8080/tcp open-webui
155+
53ff742791ae ollama/ollama "/bin/ollama serve" 29 minutes ago Up 29 minutes 0.0.0.0:11434->11434/tcp ollama
156+
```
157+
158+
Open the Web UI at http://localhost:3000
159+
160+
![image](https://github.com/user-attachments/assets/d762338d-4a95-4d89-85de-799050debe7f)
161+
162+
163+
### To add Additional Model connections
164+
165+
Then goto "Setting" -> Admin Setting -> Connection
166+
167+
![image](https://github.com/user-attachments/assets/eb05a998-a89a-4351-8587-83b8b307b0a9)
168+
169+
Then goto "Setting" -> Admin Setting -> Connection
170+
![image](https://github.com/user-attachments/assets/0e54b11f-874e-45a9-8de8-9dabd9bdb9ae)
171+
172+
173+
### Now you are all set with your Personal Chatbot of your Laptop
174+
175+
![image](https://github.com/user-attachments/assets/b1dd25c4-2f1f-48f2-9ab7-bb10b2e05e58)
176+
177+
178+
## Troubleshooting
179+
180+
- **Environment Variable Not Set**: If the application is not connecting to the Ollama backend, ensure that the `OLLAMA_IP` variable is set correctly and that the server is reachable.
181+
- **Port Issues**: Ensure that port `11434` (or the port you specified) is open and accessible.
182+
183+
184+
## Contributing
185+
186+
If you would like to contribute, please fork the repository and submit a pull request with your changes.

0 commit comments

Comments
 (0)