Add DeepHermes to list of supported models

docker-compose.yml

@@ -50,7 +50,7 @@ services:
     # need to use the .env file UNLESS config.py is modified
     container_name: node-app
     build:
-      dockerfile: Dockerfile-node-dev
+      dockerfile: napthaai/node:latest
     env_file:
       - ./.env
     volumes:

docs/advanced.md

@@ -65,7 +65,16 @@ Note that the `RMQ_*` variables will be passed in to create the default RMQ user
 
 ## Building the Docker Image
 
-The Naptha node uses a cross-platform Dockerfile intended for use with `docker buildx`. This is automatically used by Naptha's CI/CD, so when you pull the image, the right architecture will be used. To build the image with `docker buildx` yourself, you must first install Docker on your system. Once you have done that, complete the following steps to enable the `buildx` plugin:
+The Naptha node uses a platform-agnostic Dockerfile intended for use with `docker buildx`. 
+
+This is automatically used by Naptha's CI/CD, so when you pull the image from `napthaai/node:latest`, the right architecture will be available for your device, and will be pulled automatically. However, legacy `docker build` commands are automatically supported: 
+
+```shell
+docker build -t account/repository:tag -f Dockerfile-node .
+```
+
+
+To build the image with `docker buildx` yourself, you must first install Docker on your system. Once you have done that, complete the following steps to enable the `buildx` plugin:
 
 
 ```shell 
@@ -82,28 +91,33 @@ push the manifests up to your dockerhub / other container repo, since the local
 does not support manifest lists for multi-platform builds. In that case, you need to specify a full container tag of 
 `account/repository:tag`
 
+To build the image for the architecture of your machine, run:
+
+```shell 
+docker buildx build \
+  -t account/repository:tag \
+  -f Dockerfile-node \
+  . --load 
+```
+
+Or, you can use the `--platform` flag to specify one or more specific architectures to build for.
+This can be useful for example if you want to build the node on an ARM machine (e.g. a M-series Macbook) 
+and then run it on a remote server with a different architecture.
+
 ```shell
 # for ARM CPUs only:
 docker buildx build \
-  --platform linux/arm64 \
+  --platform linux/arm64 # you can specify one or more target 
   -t example-docker-tag \
-  -f buildx.Dockerfile \
-  --load . 
+  -f Dockerfile-node \
+  . --load
 
 # for multiple target platforms:
 docker buildx build \
   --platform linux/arm64,linux/amd64 \
   -t account/repository:tag \
-  -f buildx.Dockerfile \ 
-  --push .
-
+  -f Dockerfile-node \
+  . --load
+```
 
-# for GPU-accelerated inference with ollama or vLLM; use an nvidia/cuda base image. replace 12.4.1 with your CUDA version;
-# see https://hub.docker.com/r/nvidia/cuda/tags for a list of supported images.
-docker buildx build \
-    --platform linux/arm64 \
-    --build-arg BASE_IMAGE=nvidia/cuda:12.4.1-devel-ubuntu22.04 \ 
-    -t yourdockerprofile/yourrepo:yourtag \
-    -f buildx.Dockerfile \
-    --load .
-```
+Remember that with `docker buildx`, images are not automatically added to your build cache. You can use `--load` to load the image into your local docker cache, or use `--push` to push the image to a remote container repository.

docs/docker.md

@@ -38,20 +38,28 @@ The images will be pulled from `napthaai/node:latest` on Docker hub. It will not
 The docker compose runs in detached mode, so you can use `docker compose logs -f` to see the logs of the node. You can check the logs of individual services using:
 
 ```bash
-docker <service_name> logs -f
+docker logs -f <container_id>
 ```
 
-where `<service_name>` is the name of the service you want to check the logs of e.g. `node-app`, `litellm`, `node-ollama`, `Hermes-3-Llama-3.1-8B`, `node-pgvector`, etc. You can run `docker ps -a` to see the names of all the services running in docker compose.
+where `<container_id>` is the id of the service you want to check the logs of. You can get a list of all currently-running containers and their IDs by running `docker ps`. 
+Some containers in the compose configuration have human-readable `container_name` fields set, e.g. `node-app`, `litellm`, `node-ollama`, `Hermes-3-Llama-3.1-8B`, `node-pgvector`, etc. in which can you can run 
+
+```bash 
+docker logs -f <container_name>
+```
 
 ### Accessing a Service's Container
 
-To explore a service's filesystem or run commands inside a container, you can use:
+To run commands inside a container using an interactive shell, e.g. to explore the filesystem, you can use:
 
 ```bash
-docker exec -it <service_name> /bin/bash
+docker exec -it <container_id> /bin/bash
+
+# or, for some containers that don't have bash:
+docker exec -it <container_id> /bin/sh
 ```
 
-and `exit` to exit the container.
+Then, you can run `exit` to exit the shell inside the container when you are finished. This will not stop the container, it will just exit the shell.
 
 ## Stopping the Node
 
@@ -61,5 +69,9 @@ To stop the node, you can use:
 bash docker-ctl.sh down
 ```
 
+This script is _auto-generated_ by `launch.sh` and is used to stop the node and remove the containers. It will not be present in the repository until you start the node with `launch.sh`.
+
+Note that by default, this will _not_ remove persistant volumes created by certain containers, e.g. the postgres database and the rabbitMQ server. If you want to remove these volumes, you can run `docker volume ls` and `docker volume rm` to find, list, and remove the volume(s) that you want to remove. 
+
 
 

launch.sh

@@ -2077,7 +2077,7 @@ launch_docker() {
 #!/bin/bash
 case "\$1" in
     "down")
-        env \$(cat .env | grep -v '^#' | xargs) $GPU_ASSIGNMENTS docker compose -f docker-compose.yml $COMPOSE_FILES down -v
+        env \$(cat .env | grep -v '^#' | xargs) $GPU_ASSIGNMENTS docker compose -f docker-compose.yml $COMPOSE_FILES down 
         ;;
     "logs")
         env \$(cat .env | grep -v '^#' | xargs) $GPU_ASSIGNMENTS docker compose -f docker-compose.yml $COMPOSE_FILES logs -f
@@ -2096,7 +2096,7 @@ EOF
 #!/bin/bash
 case "\$1" in
     "down")
-        docker compose -f docker-compose.yml $COMPOSE_FILES down -v
+        docker compose -f docker-compose.yml $COMPOSE_FILES down
         ;;
     "logs")
         docker compose -f docker-compose.yml $COMPOSE_FILES logs -f

node/compose-files/vllm-models/DeepHermes-3-Llama-3-8B-Preview.yml

@@ -0,0 +1,33 @@
+services:
+  DeepHermes-3-Llama-3-8B-Preview:
+    image: vllm/vllm-openai:latest
+    entrypoint: [
+      "vllm", "serve", "NousResearch/DeepHermes-3-Llama-3-8B-Preview",
+      "--enable-prefix-caching", "--enable-chunked-prefill",
+      "--gpu-memory-utilization", "0.98",
+      "--max-model-len", "131072",
+      "--enable-auto-tool-choice", "--tool-call-parser", "hermes"
+    ]
+    environment:
+      HUGGING_FACE_HUB_TOKEN: ${HUGGINGFACE_TOKEN:?error}
+    volumes:
+      - type: bind
+        source: ${HF_HOME:?error}
+        target: /root/.cache/huggingface
+      - type: bind
+        source: ./node/inference/configs
+        target: /usr/app # configs like chat templates, vllm configs, tool parsers
+    ipc: host
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: "nvidia"
+              capabilities: [ "gpu" ]
+              device_ids: ["${GPU_ID_deephermes_3_llama_3_8b_preview:?error}"]
+    networks:
+      - naptha-network
+
+networks:
+  naptha-network:
+    external: true

node/inference/litellm/generate_litellm_config.py

@@ -27,7 +27,8 @@
     "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B": 2,
     "microsoft/phi-4": 1,
     "mistralai/Mistral-Small-24B-Instruct-2501": 2,
-    "Qwen/QwQ-32B-Preview": 2
+    "Qwen/QwQ-32B-Preview": 2,
+    "NousResearch/DeepHermes-3-Llama-3-8B-Preview": 1
 }
 
 if VLLM_MODELS:

node/storage/hub/hub.py

@@ -22,6 +22,7 @@ def __init__(self, *args, **kwargs):
             self.hub_url = LOCAL_HUB_URL
         else:
             self.hub_url = PUBLIC_HUB_URL
+
         self.ns = os.getenv("HUB_DB_SURREAL_NS")
         self.db = os.getenv("HUB_DB_SURREAL_NAME")