Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for deploying as a StatefulSet #116

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat: add support for deploying as a StatefulSet #116

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7b63e96
feat: add support for deploying as a StatefulSet
eric-zadara Nov 18, 2024
3a230f4
chore: bump Chart.yaml version and update artifacthub summary
eric-zadara Nov 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@ description: Get up and running with large language models locally.

type: application

version: 0.66.0
version: 0.66.1

appVersion: "0.4.2"

annotations:
artifacthub.io/category: ai-machine-learning
artifacthub.io/changes: |
- kind: changed
description: upgrade app version to 0.4.2
- kind: added
description: add support for deploying as a StatefulSet
links:
- name: Ollama release v0.4.2
url: https://github.com/ollama/ollama/releases/tag/v0.4.2
- name: add support for deploying as a StatefulSet
url: https://github.com/otwld/ollama-helm/pull/116

kubeVersion: "^1.16.0-0"
home: https://ollama.ai/
Expand Down
7 changes: 7 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,13 @@ ingress:
| ollama.insecure | bool | `false` | Add insecure flag for pulling at container startup |
| ollama.models | list | `[]` | List of models to pull at container startup The more you add, the longer the container will take to start if models are not present models: - llama2 - mistral |
| ollama.mountPath | string | `""` | Override ollama-data volume mount path, default: "/root/.ollama" |
| persistence.enabled | bool | `false` | Enable persistence using a StatefulSet This disables the Deployment in favor of a StatefulSet |
| persistence.podManagementPolicy | string | `"Parallel"` | Deploy pods in parallel or sequentially. Parallel or OrderedReady |
| persistence.size | string | `"30Gi"` | Ollama server data Persistent Volume Claim Template size used |
| persistence.storageClass | string | `""` | Ollama server data Persistent Volume Storage Class If defined, storageClassName: <storageClass> If set to "-", storageClassName: "", which disables dynamic provisioning If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. (gp2 on AWS, standard on GKE, AWS & OpenStack) |
| persistence.subPath | string | `""` | Subdirectory of Ollama server data Persistent Volume to mount Useful if the volume's root directory is not empty |
| persistence.updateStrategy | object | `{}` | Specify updateStrategy for the StatefulSet ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies |
| persistence.volumeClaimTemplates | list | `[]` | Additional volumeClaimTemplates |
| persistentVolume.accessModes | list | `["ReadWriteOnce"]` | Ollama server data Persistent Volume access modes Must match those of existing PV or dynamic provisioner Ref: http://kubernetes.io/docs/user-guide/persistent-volumes/ |
| persistentVolume.annotations | object | `{}` | Ollama server data Persistent Volume annotations |
| persistentVolume.enabled | bool | `false` | Enable persistence using PVC |
Expand Down
43 changes: 41 additions & 2 deletions templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,24 @@
{{- if not .Values.knative.enabled }}
apiVersion: apps/v1
kind: Deployment
kind: {{ if .Values.persistence.enabled }}StatefulSet{{ else }}Deployment{{ end }}
metadata:
name: {{ include "ollama.fullname" . }}
labels:
{{- include "ollama.labels" . | nindent 4 }}
spec:
{{- if .Values.persistence.enabled }}
serviceName: {{ include "ollama.fullname" . }}
{{- end }}
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
{{- if or .Values.updateStrategy.type .Values.updateStrategy.rollingUpdate }}
{{- if and .Values.persistence.enabled .Values.persistence.podManagementPolicy }}
podManagementPolicy: {{ .Values.persistence.podManagementPolicy }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add it at root values with a comment for StatefulSet only.

{{- end }}
{{- if and .Values.persistence.enabled .Values.persistence.updateStrategy }}
updateStrategy: {{- toYaml .Values.persistence.updateStrategy | nindent 4 }}
{{- end }}
{{- if and (not .Values.persistence.enabled) (or .Values.updateStrategy.type .Values.updateStrategy.rollingUpdate) }}
strategy: {{ .Values.updateStrategy | toYaml | nindent 4 }}
{{- end }}
selector:
Expand Down Expand Up @@ -109,11 +118,19 @@ spec:
{{- toYaml $ressources | nindent 12 }}
{{- end}}
volumeMounts:
{{- if or (.Values.persistentVolume.enabled) (and (not .Values.persistentVolume.enabled) (not .Values.persistence.enabled)) }}
- name: ollama-data
mountPath: {{ .Values.ollama.mountPath | default "/root/.ollama" }}
{{- if .Values.persistentVolume.subPath }}
subPath: {{ .Values.persistentVolume.subPath }}
{{- end }}
{{- else if and (.Values.persistence.enabled) (not .Values.persistentVolume.enabled) }}
- name: {{ include "ollama.fullname" . }}
mountPath: {{ .Values.ollama.mountPath | default "/root/.ollama" }}
{{- if .Values.persistence.subPath }}
subPath: {{ .Values.persistence.subPath }}
{{- end }}
{{- end }}
{{- with .Values.volumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
Expand Down Expand Up @@ -157,6 +174,7 @@ spec:
echo "{{ include "ollama.modelList" . }}" | xargs -n1 /bin/ollama pull {{ternary "--insecure" "" .Values.ollama.insecure | toString }}
{{- end }}
{{- end }}
{{- if or (.Values.persistentVolume.enabled) (and (not .Values.persistentVolume.enabled) (not .Values.persistence.enabled)) }}
volumes:
- name: ollama-data
{{- if .Values.persistentVolume.enabled }}
Expand All @@ -168,6 +186,7 @@ spec:
{{- with .Values.volumes }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
Expand Down Expand Up @@ -203,4 +222,24 @@ spec:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
{{- if and (.Values.persistence.enabled) (not .Values.persistentVolume.enabled) }}
volumeClaimTemplates:
- metadata:
name: {{ include "ollama.fullname" . }}
{{- if .Values.persistence.annotations }}
annotations: {{- toYaml .Values.persistence.annotations | nindent 10 }}
{{- end }}
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: {{ .Values.persistence.size }}
{{- if .Values.persistence.storageClass }}
storageClassName: {{ tpl .Values.persistence.storageClass . | quote }}
{{- end }}
{{- with .Values.persistence.volumeClaimTemplates }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
{{- end }}
32 changes: 32 additions & 0 deletions values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,38 @@ extraEnvFrom: []
# - configMapRef:
# name: my-env-configmap

# Enable persistence using StatefulSets
# ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
persistence:
# -- Enable persistence using a StatefulSet
# This disables the Deployment in favor of a StatefulSet
enabled: false

# -- Ollama server data Persistent Volume Claim Template size used
size: 30Gi

# -- Ollama server data Persistent Volume Storage Class
# If defined, storageClassName: <storageClass>
# If set to "-", storageClassName: "", which disables dynamic provisioning
# If undefined (the default) or set to null, no storageClassName spec is
# set, choosing the default provisioner. (gp2 on AWS, standard on
# GKE, AWS & OpenStack)
storageClass: ""

# -- Subdirectory of Ollama server data Persistent Volume to mount
# Useful if the volume's root directory is not empty
subPath: ""

# -- Additional volumeClaimTemplates
volumeClaimTemplates: []
Comment on lines +280 to +301
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can reuse the persistentVolume values to avoid duplicate code and add comment to specify to indicate whether it's for Deployment or Statefulsets.


# -- Deploy pods in parallel or sequentially. Parallel or OrderedReady
podManagementPolicy: "Parallel"

# -- Specify updateStrategy for the StatefulSet
# ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies
updateStrategy: {}

# Enable persistence using Persistent Volume Claims
# ref: https://kubernetes.io/docs/concepts/storage/persistent-volumes/
persistentVolume:
Expand Down