Skip to content

Commit

Permalink
Realigned readme
Browse files Browse the repository at this point in the history
Signed-off-by: Florian Klinger <[email protected]>
  • Loading branch information
nc-fkl committed Feb 29, 2024
1 parent 1bda720 commit f4d603c
Showing 1 changed file with 36 additions and 3 deletions.
39 changes: 36 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,13 @@
1. `python -m venv .venv`
2. `. .venv/bin/activate`
3. Install requirements

3.1 For using CPU:

`pip install --no-deps -r requirements.cpu.txt`

3.2 Using GPU with CUDA:

`pip install --no-deps -r requirements.cuda.txt`
4. Install pandoc from your desired package manager (`# apt install pandoc` for Debian-based systems)
5. Copy example.env to .env and fill in the variables
Expand All @@ -38,16 +42,20 @@
## Complex Install (with docker)
1. Build the image
*(this is a good place to edit the example.env file before building the container)*

1.1 **CPU:**

`docker build -t context_chat_backend_dev . -f Dockerfile.cpu`

1.2 **GPU:**

`docker build -t context_chat_backend_dev . -f Dockerfile.cuda11.8`

2. `docker run --add-host=host.docker.internal:host-gateway -p 10034:10034 context_chat_backend_dev`
3. Volumes can be mounted for `model_files` and `vector_db_files` if you wish with `-v $(pwd)/model_files:/app/model_files` and similar for vector_db_files
4. If your Nextcloud is running inside a docker container, ensure you have mounted the docker socket inside your container and has the correct permissions for the web server user to have access to it or add the web server to the docker group:
- for docker compose
```yaml
```
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
```
Expand All @@ -61,9 +69,13 @@ volumes:
1. `python -m venv .venv`
2. `. .venv/bin/activate`
3. Install requirements
4.
3.1 **CPU:**

`pip install --no-deps -r requirements.cpu.txt`

3.2 **GPU:**

`pip install --no-deps -r requirements.cuda.txt`

5. Install pandoc from your desired package manager (`# apt install pandoc` for Debian-based systems)
Expand All @@ -75,10 +87,15 @@ volumes:
## Manual Install (with docker)
1. Build the image
*(this is a good place to edit the example.env file before building the container)*

1.1 **CPU:**

`docker build -t context_chat_backend_dev . -f Dockerfile.cpu`

1.2 **GPU:**

`docker build -t context_chat_backend_dev . -f Dockerfile.cuda11.8`

2. `docker run --add-host=host.docker.internal:host-gateway -p10034:10034 context_chat_backend_dev`
3. Volumes can be mounted for `model_files` and `vector_db_files` if you wish with `-v $(pwd)/model_files:/app/model_files` and similar for vector_db_files
4. If your Nextcloud is running inside a docker container, there are two ways to configure the deploy daemon
Expand Down Expand Up @@ -114,7 +131,7 @@ An alternative method would be to provide the Nextcloud's web server user access
Mount the docker.sock in the Nextcloud container if you happen to use a containerized install of Nextcloud and ensure correct permissions for the web server user to access it.

- for docker compose
```yaml
```
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
```
Expand All @@ -134,16 +151,23 @@ docker build --no-cache -f Dockerfile.cuda11.8 -t context_chat_backend_dev:11.8
- ***Parameter explanation:***

`--no-cache`

Tells Docker to build the image without using any cache from previous builds.


`-f Dockerfile.cuda11.8`
The `-f` or `--file` option specifies the name of the Dockerfile to use for the build. In this case, `Dockerfile.cuda11.8`

The *-f* or *--file* option specifies the name of the Dockerfile to use for the build. In this case *Dockerfile.cuda11.8*


`-t context_chat_backend_dev:11.8`

The *-t* or *--tag* option allows you to name and optionally tag your image, so you can refer to it later.
In this case we name it *context_chat_backend_dev*with the specified version *11.8*


`.`

This final argument specifies the build context to the Docker daemon. In most cases, it's the path to a directory containing the Dockerfile and any other files needed for the build. Using `.` means "use the current directory as the build context."

**2. Run the image**
Expand All @@ -168,30 +192,39 @@ docker run \
- ***Parameter explanation:***

`-v ./config.cuda.yaml:/app/config.yaml`

Mounts the config_cuda.yaml which will be used inside the running image

`-v ./context_chat_backend:/app/context_chat_backend`

Mounts the context_chat_backend into the docker image

`-v /var/run/docker.sock:/var/run/docker.sock`

Mounts the Docker socket file from the host into the container. This is done to allow the Docker client running inside the container to communicate with the Docker daemon on the host, essentially controlling Docker and GPU from within the container.

`-v ./persistent_storage:/app/persistent_storage`

Mounts the persistent storage into the docker instance to keep downloaded models stored for the future.

`--env-file example.env`

Specifies an environment file named example.env to load environment variables from. Please adjust it for your needs.

`-p 10034:10034`

This publishes a container's port (10034) to the host (10034). Please align it with your environment file

`-e CUDA_VISIBLE_DEVICES=0`

Used to limit which GPUs are visible to CUDA applications running in the container. In this case, it restricts visibility to only the first GPU.

`--gpus all`

Grants the container access to all GPUs available on the host. This is crucial for running GPU-accelerated applications inside the container.

`context_chat_backend_dev:11.8`

Specifies the image to use for creating the container. In this case we have build the image in 1.) with the specified tag

**3. Register context_chat_backend**
Expand Down

0 comments on commit f4d603c

Please sign in to comment.