From 480c4763543a701e738f4f2a1e7a62784746d88d Mon Sep 17 00:00:00 2001 From: bstuder99 <81711774+bstuder99@users.noreply.github.com> Date: Wed, 21 Feb 2024 12:30:11 -0500 Subject: [PATCH 1/2] adjsutments --- architecture/kubecost-cluster-roles.md | 104 +++++++++++++++++- code-contribution.md | 2 +- .../custom-prom/gcp-gmp-integration.md | 2 +- .../windows-node-support.md | 35 +++++- .../thanos-setup/thanos-setup.md | 2 +- troubleshooting/http-troubleshooting.md | 6 +- 6 files changed, 143 insertions(+), 8 deletions(-) diff --git a/architecture/kubecost-cluster-roles.md b/architecture/kubecost-cluster-roles.md index 694e278bb..1dbebc6fd 100644 --- a/architecture/kubecost-cluster-roles.md +++ b/architecture/kubecost-cluster-roles.md @@ -6,4 +6,106 @@ Kubecost requires read only RBAC permissions on most cluster resources in order Kubecost requires `get`, `list`, and `watch` permissions over many common Kubernetes pod and pod controller resources such as pods, deployments, StatefulSets as well as other resources which factor into to cost analysis such as namespaces, nodes, events, etc. -The source of these rules can be found in [Kubecost's ClusterRole template](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/templates/cost-analyzer-cluster-role-template.yaml). +The source of these rules can be found in Kubecost's ClusterRole template: + +``` +{{- if not .Values.kubecostModel.etlReadOnlyMode -}} +{{- if and .Values.reporting .Values.reporting.logCollection -}} +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + namespace: {{ .Release.Namespace }} + name: {{ template "cost-analyzer.serviceAccountName" . }} + labels: + {{ include "cost-analyzer.commonLabels" . | nindent 4 }} +rules: +- apiGroups: + - '' + resources: + - "pods/log" + verbs: + - get + - list + - watch +--- +{{- end }} +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ template "cost-analyzer.serviceAccountName" . }} + labels: + {{ include "cost-analyzer.commonLabels" . | nindent 4 }} +rules: + - apiGroups: + - '' + resources: + - configmaps + - nodes + - pods + - events + - services + - resourcequotas + - replicationcontrollers + - limitranges + - persistentvolumeclaims + - persistentvolumes + - namespaces + - endpoints + verbs: + - get + - list + - watch + - apiGroups: + - apps + resources: + - statefulsets + - deployments + - daemonsets + - replicasets + verbs: + - get + - list + - watch + - apiGroups: + - batch + resources: + - cronjobs + - jobs + verbs: + - get + - list + - watch + - apiGroups: + - autoscaling + resources: + - horizontalpodautoscalers + verbs: + - get + - list + - watch + - apiGroups: + - policy + resources: + - poddisruptionbudgets + verbs: + - get + - list + - watch + - apiGroups: + - storage.k8s.io + resources: + - storageclasses + verbs: + - get + - list + - watch + - apiGroups: + - events.k8s.io + resources: + - events + verbs: + - get + - list + - watch +{{- end }} +``` diff --git a/code-contribution.md b/code-contribution.md index b4d36bbac..f4ff5eb0c 100644 --- a/code-contribution.md +++ b/code-contribution.md @@ -11,4 +11,4 @@ Kubecost offers a number of open-source software projects and tools which allow * [kubectl-cost](https://github.com/kubecost/kubectl-cost) * [Cluster Turndown](https://github.com/kubecost/cluster-turndown) -The core of Kubecost is based on the open source CNCF project [OpenCost](https://www.opencost.io/). For contribution guidelines to OpenCost, please see [here](https://github.com/opencost/opencost/blob/develop/CONTRIBUTING.md). +The core of Kubecost is based on the open source CNCF project [OpenCost](https://www.opencost.io/). Read OpenCost's [contribution guide](https://github.com/opencost/opencost/blob/develop/CONTRIBUTING.md) for more info. diff --git a/install-and-configure/advanced-configuration/custom-prom/gcp-gmp-integration.md b/install-and-configure/advanced-configuration/custom-prom/gcp-gmp-integration.md index 0caa78e3c..17ba90c73 100644 --- a/install-and-configure/advanced-configuration/custom-prom/gcp-gmp-integration.md +++ b/install-and-configure/advanced-configuration/custom-prom/gcp-gmp-integration.md @@ -48,7 +48,7 @@ In this installation command, these additional flags are added to have Kubecost * `global.gmp.enabled` and `global.gmp.gmpProxy.projectId` are for enabling the GMP integration. * `prometheus.server.global.external_labels.cluster_id` and `kubecostProductConfigs.clusterName` helps to set the name for your Kubecost setup. -You can find additional configurations at our main [values.yaml](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values.yaml) file. +You can find additional configurations at our main [*values.yaml*](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values.yaml) file. Your Kubecost setup now writes and collects data from GMP. Data should be ready for viewing within 15 minutes. diff --git a/install-and-configure/advanced-configuration/windows-node-support.md b/install-and-configure/advanced-configuration/windows-node-support.md index 069d65eb2..23107514a 100644 --- a/install-and-configure/advanced-configuration/windows-node-support.md +++ b/install-and-configure/advanced-configuration/windows-node-support.md @@ -41,7 +41,40 @@ For DaemonSets, set the affinity to only allow scheduling on Windows nodes: - linux ``` -See the list of all deployments and DaemonSets in this [*values-windows-node-affinity.yaml*](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values-windows-node-affinity.yaml) file. +See the list of all deployments and DaemonSets in this [*values-windows-node-affinity.yaml*](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values-windows-node-affinity.yaml) file: + +``` +kubecostMetrics: + exporter: + nodeSelector: + kubernetes.io/os: linux + +nodeSelector: + kubernetes.io/os: linux + +networkCosts: + nodeSelector: + kubernetes.io/os: linux + +prometheus: + server: + nodeSelector: + kubernetes.io/os: linux + nodeExporter: + enabled: true + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: kubernetes.io/os + operator: In + values: + - linux +grafana: + nodeSelector: + kubernetes.io/os: linux +``` ## Metrics diff --git a/install-and-configure/install/multi-cluster/thanos-setup/thanos-setup.md b/install-and-configure/install/multi-cluster/thanos-setup/thanos-setup.md index 512a07832..e06def625 100644 --- a/install-and-configure/install/multi-cluster/thanos-setup/thanos-setup.md +++ b/install-and-configure/install/multi-cluster/thanos-setup/thanos-setup.md @@ -26,7 +26,7 @@ Consider the following Thanos recommendations for secondaries: * Optionally deploy `thanos-bucket` in each additional cluster, but it is not required. * Optionally disable `thanos.store` and `thanos.query` (Clusters with store/query disabled will only have access to their metrics but will still write to the global bucket.) - Thanos modules can be disabled in [thanos/values.yaml](https://github.com/kubecost/cost-analyzer-helm-chart/blob/master/cost-analyzer/charts/thanos/values.yaml), or in [values-thanos.yaml](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values-thanos.yaml) if overriding these values from a values-thanos.yaml file supplied from the command line (`helm upgrade kubecost -f values.yaml -f values-thanos.yaml`), or by passing these parameters directly via Helm install or upgrade as follows: + Thanos modules can be disabled in [*thanos/values.yaml*](https://github.com/kubecost/cost-analyzer-helm-chart/blob/master/cost-analyzer/charts/thanos/values.yaml), or in [*values-thanos.yaml*](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values-thanos.yaml) if overriding these values from a values-thanos.yaml file supplied from the command line (`helm upgrade kubecost -f values.yaml -f values-thanos.yaml`), or by passing these parameters directly via Helm install or upgrade as follows: ``` --set thanos.compact.enabled=false --set thanos.bucket.enabled=false diff --git a/troubleshooting/http-troubleshooting.md b/troubleshooting/http-troubleshooting.md index 9051a7c57..dfea7f0a2 100644 --- a/troubleshooting/http-troubleshooting.md +++ b/troubleshooting/http-troubleshooting.md @@ -10,7 +10,7 @@ This is most likely due to a user authenticating to Kubecost or performing an ac * Review the Helm values used to determine if SAML, OIDC, and RBAC are being used which could lead to the 403 error. * Verify if the 403 error occurs when port forwarding to the cost-analyzer frontend. This will allow you to determine if the error is being introduced by an external service. -* Ensure that [readonly has not been set to `true`](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values.yaml#L888)in your values.yaml file +* Ensure that [readonly has not been set to `true`](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values.yaml#L888) in your *values.yaml* file ## HTTP 499 Client closed connection @@ -22,9 +22,9 @@ This is usually the NGINX gateway (inside the `kubecost-cost-analyzer` pod) repo ### Test command -> **Note**: The following test command can be used for troubleshooting both 499 and 504 errors. +The following test command can be used for troubleshooting both 499 and 504 errors. -* If running the following command fails or hangs when the pod is ready, the error is likely due to intermittent DNS: +* reIf running the following command fails or hangs when the pod is ready, the error is likely due to intermittent DNS: * `kubectl exec -i -t -n kubecost kubecost-cost-analyzer-55c45d9d95-8m2sq -c cost-analyzer-frontend -- curl kubecost-cost-analyzer.kubecost:9090/model/clusterInfo` ## HTTP 504 Gateway timeout From f29a168da805b613dbd1c8ee7a3f27f6a5c75df5 Mon Sep 17 00:00:00 2001 From: bstuder99 <81711774+bstuder99@users.noreply.github.com> Date: Wed, 21 Feb 2024 12:48:24 -0500 Subject: [PATCH 2/2] Fix --- troubleshooting/http-troubleshooting.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/troubleshooting/http-troubleshooting.md b/troubleshooting/http-troubleshooting.md index dfea7f0a2..e31a31e50 100644 --- a/troubleshooting/http-troubleshooting.md +++ b/troubleshooting/http-troubleshooting.md @@ -10,7 +10,13 @@ This is most likely due to a user authenticating to Kubecost or performing an ac * Review the Helm values used to determine if SAML, OIDC, and RBAC are being used which could lead to the 403 error. * Verify if the 403 error occurs when port forwarding to the cost-analyzer frontend. This will allow you to determine if the error is being introduced by an external service. -* Ensure that [readonly has not been set to `true`](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values.yaml#L888) in your *values.yaml* file +* Ensure that [`readonly` has not been set to `true`](https://github.com/kubecost/cost-analyzer-helm-chart/blob/develop/cost-analyzer/values.yaml) in your *values.yaml* file: + +``` +## Disable updates to kubecost from the frontend UI and via POST request +## +# readonly: false +``` ## HTTP 499 Client closed connection