6
6
- Docker installed on your system
7
7
- Basic knowledge of Docker commands
8
8
9
- ## 1. Build the Streamlit Chatbot Docker Image
9
+ ## Build the Streamlit Chatbot Docker Image
10
10
11
11
### Build the Docker Image
12
12
@@ -16,26 +16,31 @@ Run the following command in the same directory as the `Dockerfile`:
16
16
docker build -t ai-chatbot .
17
17
```
18
18
19
- ## 2. Run Ollama on Docker
19
+ ## Run Ollama on Docker
20
20
21
21
To run Ollama with the LLaMA 3.2:1B model, execute:
22
22
23
23
``` sh
24
24
docker run -d --name ollama -p 11434:11434 ollama/ollama:latest
25
25
```
26
+ If you have GPU in your machine then use
27
+
28
+ ``` sh
29
+ docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
30
+ ```
26
31
27
32
Pull and prepare the LLaMA 3.2:1B model:
28
33
29
34
``` sh
30
35
docker exec -it ollama ollama pull llama3.2:1b
31
36
```
32
37
33
- ## 3. Run the Streamlit Chatbot Container
38
+ ## Run the Streamlit Chatbot Container
34
39
35
40
Once Ollama is running, start the chatbot container:
36
41
37
42
``` sh
38
- docker run -d --name ai-chatbot -p 8501:8501 streamlit -chatbot
43
+ docker run -d --name ai-chatbot -p 8501:8501 ai -chatbot
39
44
```
40
45
41
46
Get the Container IP of the Ollama Container
@@ -57,7 +62,7 @@ On Linux/Mac
57
62
docker inspect 2cf4d51a43c9 | grep IPAddress
58
63
```
59
64
60
- ## 4. Access the Chatbot
65
+ ## Access the Chatbot
61
66
62
67
Open your browser and visit:
63
68
@@ -66,7 +71,23 @@ http://localhost:8501
66
71
```
67
72
Update the Backend URL
68
73
69
- ## 5. Stop and Remove Containers
74
+ ### Check if Ollama is serving your model
75
+
76
+ For CPU Only
77
+ ```
78
+ docker exec ollama ollama ps
79
+ NAME ID SIZE PROCESSOR UNTIL
80
+ llama3.2:1b baf6a787fdff 2.2 GB 100% CPU 4 minutes from now
81
+ ```
82
+ With GPU support
83
+
84
+ ```
85
+ docker exec ollama ollama ps
86
+ NAME ID SIZE PROCESSOR UNTIL
87
+ llama3.2:1b baf6a787fdff 2.7 GB 100% GPU 4 minutes from now
88
+ ```
89
+
90
+ ## Stop and Remove Containers
70
91
71
92
To stop and remove all running containers:
72
93
0 commit comments