Skip to content

Commit f4d603c

Browse files
committed
Realigned readme
Signed-off-by: Florian Klinger <[email protected]>
1 parent 1bda720 commit f4d603c

File tree

1 file changed

+36
-3
lines changed

1 file changed

+36
-3
lines changed

README.md

Lines changed: 36 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,13 @@
2525
1. `python -m venv .venv`
2626
2. `. .venv/bin/activate`
2727
3. Install requirements
28+
2829
3.1 For using CPU:
30+
2931
`pip install --no-deps -r requirements.cpu.txt`
32+
3033
3.2 Using GPU with CUDA:
34+
3135
`pip install --no-deps -r requirements.cuda.txt`
3236
4. Install pandoc from your desired package manager (`# apt install pandoc` for Debian-based systems)
3337
5. Copy example.env to .env and fill in the variables
@@ -38,16 +42,20 @@
3842
## Complex Install (with docker)
3943
1. Build the image
4044
*(this is a good place to edit the example.env file before building the container)*
45+
4146
1.1 **CPU:**
47+
4248
`docker build -t context_chat_backend_dev . -f Dockerfile.cpu`
49+
4350
1.2 **GPU:**
51+
4452
`docker build -t context_chat_backend_dev . -f Dockerfile.cuda11.8`
4553

4654
2. `docker run --add-host=host.docker.internal:host-gateway -p 10034:10034 context_chat_backend_dev`
4755
3. Volumes can be mounted for `model_files` and `vector_db_files` if you wish with `-v $(pwd)/model_files:/app/model_files` and similar for vector_db_files
4856
4. If your Nextcloud is running inside a docker container, ensure you have mounted the docker socket inside your container and has the correct permissions for the web server user to have access to it or add the web server to the docker group:
4957
- for docker compose
50-
```yaml
58+
```
5159
volumes:
5260
- /var/run/docker.sock:/tmp/docker.sock:ro
5361
```
@@ -61,9 +69,13 @@ volumes:
6169
1. `python -m venv .venv`
6270
2. `. .venv/bin/activate`
6371
3. Install requirements
72+
4.
6473
3.1 **CPU:**
74+
6575
`pip install --no-deps -r requirements.cpu.txt`
76+
6677
3.2 **GPU:**
78+
6779
`pip install --no-deps -r requirements.cuda.txt`
6880

6981
5. Install pandoc from your desired package manager (`# apt install pandoc` for Debian-based systems)
@@ -75,10 +87,15 @@ volumes:
7587
## Manual Install (with docker)
7688
1. Build the image
7789
*(this is a good place to edit the example.env file before building the container)*
90+
7891
1.1 **CPU:**
92+
7993
`docker build -t context_chat_backend_dev . -f Dockerfile.cpu`
94+
8095
1.2 **GPU:**
96+
8197
`docker build -t context_chat_backend_dev . -f Dockerfile.cuda11.8`
98+
8299
2. `docker run --add-host=host.docker.internal:host-gateway -p10034:10034 context_chat_backend_dev`
83100
3. Volumes can be mounted for `model_files` and `vector_db_files` if you wish with `-v $(pwd)/model_files:/app/model_files` and similar for vector_db_files
84101
4. If your Nextcloud is running inside a docker container, there are two ways to configure the deploy daemon
@@ -114,7 +131,7 @@ An alternative method would be to provide the Nextcloud's web server user access
114131
Mount the docker.sock in the Nextcloud container if you happen to use a containerized install of Nextcloud and ensure correct permissions for the web server user to access it.
115132

116133
- for docker compose
117-
```yaml
134+
```
118135
volumes:
119136
- /var/run/docker.sock:/var/run/docker.sock:ro
120137
```
@@ -134,16 +151,23 @@ docker build --no-cache -f Dockerfile.cuda11.8 -t context_chat_backend_dev:11.8
134151
- ***Parameter explanation:***
135152

136153
`--no-cache`
154+
137155
Tells Docker to build the image without using any cache from previous builds.
138156

157+
139158
`-f Dockerfile.cuda11.8`
140-
The `-f` or `--file` option specifies the name of the Dockerfile to use for the build. In this case, `Dockerfile.cuda11.8`
159+
160+
The *-f* or *--file* option specifies the name of the Dockerfile to use for the build. In this case *Dockerfile.cuda11.8*
161+
141162

142163
`-t context_chat_backend_dev:11.8`
164+
143165
The *-t* or *--tag* option allows you to name and optionally tag your image, so you can refer to it later.
144166
In this case we name it *context_chat_backend_dev*with the specified version *11.8*
145167

168+
146169
`.`
170+
147171
This final argument specifies the build context to the Docker daemon. In most cases, it's the path to a directory containing the Dockerfile and any other files needed for the build. Using `.` means "use the current directory as the build context."
148172

149173
**2. Run the image**
@@ -168,30 +192,39 @@ docker run \
168192
- ***Parameter explanation:***
169193

170194
`-v ./config.cuda.yaml:/app/config.yaml`
195+
171196
Mounts the config_cuda.yaml which will be used inside the running image
172197

173198
`-v ./context_chat_backend:/app/context_chat_backend`
199+
174200
Mounts the context_chat_backend into the docker image
175201

176202
`-v /var/run/docker.sock:/var/run/docker.sock`
203+
177204
Mounts the Docker socket file from the host into the container. This is done to allow the Docker client running inside the container to communicate with the Docker daemon on the host, essentially controlling Docker and GPU from within the container.
178205

179206
`-v ./persistent_storage:/app/persistent_storage`
207+
180208
Mounts the persistent storage into the docker instance to keep downloaded models stored for the future.
181209

182210
`--env-file example.env`
211+
183212
Specifies an environment file named example.env to load environment variables from. Please adjust it for your needs.
184213

185214
`-p 10034:10034`
215+
186216
This publishes a container's port (10034) to the host (10034). Please align it with your environment file
187217

188218
`-e CUDA_VISIBLE_DEVICES=0`
219+
189220
Used to limit which GPUs are visible to CUDA applications running in the container. In this case, it restricts visibility to only the first GPU.
190221

191222
`--gpus all`
223+
192224
Grants the container access to all GPUs available on the host. This is crucial for running GPU-accelerated applications inside the container.
193225

194226
`context_chat_backend_dev:11.8`
227+
195228
Specifies the image to use for creating the container. In this case we have build the image in 1.) with the specified tag
196229

197230
**3. Register context_chat_backend**

0 commit comments

Comments
 (0)