Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add service monitor to Thanos Query #100

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions api/v1alpha1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,10 @@ type CommonThanosFields struct {
// +kubebuilder:default:=logfmt
// +kubebuilder:validation:Optional
LogFormat *string `json:"logFormat,omitempty"`
// Enable self monitoring for the Thanos component.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also mention Prometheus Operator version number in comment here?

// +kubebuilder:validation:Optional
// +kubebuilder:default:=true
EnableSelfMonitor *bool `json:"enableSelfMonitor,omitempty"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interested to hear with @saswatamcode thinks, but I think this should be its own struct because there are certain requirements for how service monitors can be created and where they should live to make this useful (thinking rhobs for example).

It could look something like this

type ServiceMonitorConfig struct {
// default true 
Enabled
Labels map[string]string
// Can leave this out for now but it might be required or we can just hard default to current for now
Namespace
// 
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's have it in separate struct and maybe as part of commonthanosfields?

}

type Additional struct {
Expand Down
5 changes: 5 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import (
"net/http/pprof"
"os"

monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1"

"github.com/prometheus/client_golang/prometheus"
versioncollector "github.com/prometheus/client_golang/prometheus/collectors/version"

Expand Down Expand Up @@ -57,6 +59,7 @@ func init() {

utilruntime.Must(monitoringthanosiov1alpha1.AddToScheme(scheme))
//+kubebuilder:scaffold:scheme
utilruntime.Must(monitoringv1.AddToScheme(scheme))
}

func main() {
Expand Down
4 changes: 4 additions & 0 deletions config/crd/bases/monitoring.thanos.io_thanoscompacts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,10 @@ spec:
pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$
type: string
type: object
enableSelfMonitor:
default: true
description: Enable self monitoring for the Thanos component.
type: boolean
groupConfig:
description: GroupConfig defines settings for group handling.
properties:
Expand Down
8 changes: 8 additions & 0 deletions config/crd/bases/monitoring.thanos.io_thanosqueries.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3574,6 +3574,10 @@ spec:
type: object
type: object
x-kubernetes-map-type: atomic
enableSelfMonitor:
default: true
description: Enable self monitoring for the Thanos component.
type: boolean
image:
description: Container image to use for the Thanos components.
type: string
Expand Down Expand Up @@ -7158,6 +7162,10 @@ spec:
default: true
description: CompressResponses enables response compression
type: boolean
enableSelfMonitor:
default: true
description: Enable self monitoring for the Thanos component.
type: boolean
image:
description: Container image to use for the Thanos components.
type: string
Expand Down
8 changes: 8 additions & 0 deletions config/crd/bases/monitoring.thanos.io_thanosreceives.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3571,6 +3571,10 @@ spec:
- key
type: object
x-kubernetes-map-type: atomic
enableSelfMonitor:
default: true
description: Enable self monitoring for the Thanos component.
type: boolean
hashrings:
description: Hashrings is a list of hashrings to route to.
items:
Expand Down Expand Up @@ -7310,6 +7314,10 @@ spec:
- name
type: object
type: array
enableSelfMonitor:
default: true
description: Enable self monitoring for the Thanos component.
type: boolean
externalLabels:
additionalProperties:
type: string
Expand Down
4 changes: 4 additions & 0 deletions config/crd/bases/monitoring.thanos.io_thanosrulers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3561,6 +3561,10 @@ spec:
- key
type: object
x-kubernetes-map-type: atomic
enableSelfMonitor:
default: true
description: Enable self monitoring for the Thanos component.
type: boolean
evaluationInterval:
default: 1m
description: EvaluationInterval is the default interval at which rules
Expand Down
4 changes: 4 additions & 0 deletions config/crd/bases/monitoring.thanos.io_thanosstores.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3551,6 +3551,10 @@ spec:
- key
type: object
x-kubernetes-map-type: atomic
enableSelfMonitor:
default: true
description: Enable self monitoring for the Thanos component.
type: boolean
ignoreDeletionMarksDelay:
default: 24h
description: |-
Expand Down
12 changes: 12 additions & 0 deletions config/rbac/role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,18 @@ rules:
- get
- list
- watch
- apiGroups:
- monitoring.coreos.com
resources:
- servicemonitors
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- monitoring.thanos.io
resources:
Expand Down
14 changes: 8 additions & 6 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ require (
github.com/imdario/mergo v0.3.16
github.com/onsi/ginkgo/v2 v2.20.2
github.com/onsi/gomega v1.34.2
github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.76.1
github.com/prometheus/client_golang v1.20.3
github.com/prometheus/common v0.59.1
github.com/prometheus/prometheus v0.54.1
Expand All @@ -32,7 +33,8 @@ require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/dennwc/varint v1.0.0 // indirect
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
github.com/emicklei/go-restful/v3 v3.12.1 // indirect
github.com/evanphx/json-patch v5.9.0+incompatible // indirect
github.com/evanphx/json-patch/v5 v5.9.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
Expand All @@ -41,9 +43,9 @@ require (
github.com/go-logfmt/logfmt v0.6.0 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-logr/zapr v1.3.0 // indirect
github.com/go-openapi/jsonpointer v0.20.2 // indirect
github.com/go-openapi/jsonreference v0.20.4 // indirect
github.com/go-openapi/swag v0.22.9 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v5 v5.2.1 // indirect
Expand Down Expand Up @@ -93,7 +95,7 @@ require (
golang.org/x/sys v0.24.0 // indirect
golang.org/x/term v0.23.0 // indirect
golang.org/x/text v0.17.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/time v0.6.0 // indirect
golang.org/x/tools v0.24.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240708141625-4ad9e859172b // indirect
Expand All @@ -105,7 +107,7 @@ require (
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apiextensions-apiserver v0.31.0 // indirect
k8s.io/klog/v2 v2.130.1 // indirect
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 // indirect
k8s.io/kube-openapi v0.0.0-20240808142205-8e686545bdb8 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
Expand Down
30 changes: 16 additions & 14 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,8 @@ github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/edsrzf/mmap-go v1.1.0 h1:6EUwBLQ/Mcr1EYLE4Tn1VdW1A4ckqCQWZBw8Hr0kjpQ=
github.com/edsrzf/mmap-go v1.1.0/go.mod h1:19H/e8pUPLicwkyNgOykDXkJ9F0MHE+Z52B8EIth78Q=
github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g=
github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/emicklei/go-restful/v3 v3.12.1 h1:PJMDIM/ak7btuL8Ex0iYET9hxM3CI2sjZtzpL63nKAU=
github.com/emicklei/go-restful/v3 v3.12.1/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
Expand All @@ -108,8 +108,8 @@ github.com/envoyproxy/go-control-plane v0.12.0/go.mod h1:ZBTaoJ23lqITozF0M6G4/Ir
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/envoyproxy/protoc-gen-validate v1.0.4 h1:gVPz/FMfvh57HdSJQyvBtF00j8JU4zdyUgIUNhlgg0A=
github.com/envoyproxy/protoc-gen-validate v1.0.4/go.mod h1:qys6tmnRsYrQqIhm2bvKZH4Blx/1gTIZ2UKVY1M+Yew=
github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U=
github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch v5.9.0+incompatible h1:fBXyNpNMuTTDdquAq/uisOr2lShz4oaXpDTX2bLe7ls=
github.com/evanphx/json-patch v5.9.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch/v5 v5.9.0 h1:kcBlZQbplgElYIlo/n1hJbls2z/1awpXxpRi0/FOJfg=
github.com/evanphx/json-patch/v5 v5.9.0/go.mod h1:VNkHZ/282BpEyt/tObQO8s5CMPmYYq14uClGH4abBuQ=
github.com/facette/natsort v0.0.0-20181210072756-2cd4dd1e2dcb h1:IT4JYU7k4ikYg1SCxNI1/Tieq/NFvh6dzLdgi7eu0tM=
Expand Down Expand Up @@ -142,12 +142,12 @@ github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-logr/zapr v1.3.0 h1:XGdV8XW8zdwFiwOA2Dryh1gj2KRQyOOoNmBy4EplIcQ=
github.com/go-logr/zapr v1.3.0/go.mod h1:YKepepNBd1u/oyhd/yQmtjVXmm9uML4IXUgMOwR8/Gg=
github.com/go-openapi/jsonpointer v0.20.2 h1:mQc3nmndL8ZBzStEo3JYF8wzmeWffDH4VbXz58sAx6Q=
github.com/go-openapi/jsonpointer v0.20.2/go.mod h1:bHen+N0u1KEO3YlmqOjTT9Adn1RfD91Ar825/PuiRVs=
github.com/go-openapi/jsonreference v0.20.4 h1:bKlDxQxQJgwpUSgOENiMPzCTBVuc7vTdXSSgNeAhojU=
github.com/go-openapi/jsonreference v0.20.4/go.mod h1:5pZJyJP2MnYCpoeoMAql78cCHauHj0V9Lhc506VOpw4=
github.com/go-openapi/swag v0.22.9 h1:XX2DssF+mQKM2DHsbgZK74y/zj4mo9I99+89xUmuZCE=
github.com/go-openapi/swag v0.22.9/go.mod h1:3/OXnFfnMAwBD099SwYRk7GD3xOrr1iL7d/XNLXVVwE=
github.com/go-openapi/jsonpointer v0.21.0 h1:YgdVicSA9vH5RiHs9TZW5oyafXZFc6+2Vc1rr/O9oNQ=
github.com/go-openapi/jsonpointer v0.21.0/go.mod h1:IUyH9l/+uyhIYQ/PXVA41Rexl+kOkAPDdXEYns6fzUY=
github.com/go-openapi/jsonreference v0.21.0 h1:Rs+Y7hSXT83Jacb7kFyjn4ijOuVGSvOdF2+tg1TRrwQ=
github.com/go-openapi/jsonreference v0.21.0/go.mod h1:LmZmgsrTkVg9LG4EaHeY8cBDslNPMo06cago5JNLkm4=
github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE=
github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=
github.com/go-resty/resty/v2 v2.13.1 h1:x+LHXBI2nMB1vqndymf26quycC4aggYJ7DECYbiz03g=
github.com/go-resty/resty/v2 v2.13.1/go.mod h1:GznXlLxkq6Nh4sU59rPmUw3VtgpO3aS96ORAI6Q7d+0=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
Expand Down Expand Up @@ -356,6 +356,8 @@ github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINE
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.76.1 h1:QU2cs0xxKYvF1JfibP/8vs+pFy6OvIpqNR2lYC4jYNU=
github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.76.1/go.mod h1:Rd8YnCqz+2FYsiGmE2DMlaLjQRB4v2jFNnzCt9YY4IM=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
Expand Down Expand Up @@ -590,8 +592,8 @@ golang.org/x/text v0.17.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/time v0.6.0 h1:eTDhh4ZXt5Qf0augr54TN6suAUudPcawVZeIAPU7D4U=
golang.org/x/time v0.6.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
Expand Down Expand Up @@ -766,8 +768,8 @@ k8s.io/client-go v0.31.0 h1:QqEJzNjbN2Yv1H79SsS+SWnXkBgVu4Pj3CJQgbx0gI8=
k8s.io/client-go v0.31.0/go.mod h1:Y9wvC76g4fLjmU0BA+rV+h2cncoadjvjjkkIGoTLcGU=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 h1:BZqlfIlq5YbRMFko6/PM7FjZpUb45WallggurYhKGag=
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340/go.mod h1:yD4MZYeKMBwQKVht279WycxKyM84kkAx2DPrTXaeb98=
k8s.io/kube-openapi v0.0.0-20240808142205-8e686545bdb8 h1:1Wof1cGQgA5pqgo8MxKPtf+qN6Sh/0JzznmeGPm1HnE=
k8s.io/kube-openapi v0.0.0-20240808142205-8e686545bdb8/go.mod h1:Os6V6dZwLNii3vxFpxcNaTmH8LJJBkOTg1N0tOA0fvA=
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 h1:pUdcCO1Lk/tbT5ztQWOBi5HBgbBP1J8+AsQnQCKsi8A=
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
Expand Down
11 changes: 10 additions & 1 deletion internal/controller/suite_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import (
"runtime"
"testing"

monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1"

. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/prometheus/client_golang/prometheus"
Expand Down Expand Up @@ -64,7 +66,11 @@ var _ = BeforeSuite(func() {

By("bootstrapping test environment")
testEnv = &envtest.Environment{
CRDDirectoryPaths: []string{filepath.Join("..", "..", "config", "crd", "bases")},
CRDDirectoryPaths: []string{
filepath.Join("..", "..", "config", "crd", "bases"),
filepath.Join("..", "..", "test", "configs", "service-monitor.yaml"),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think i misled you here because i thought that we already had those resources locally :/

a better test indeed might be in e2e - we could simply, in the first step, check against the prometheus to see if it reports the up metric for the component.

after deleting the SM we can just check that its gone. My apologies there I didnt check thoroughly enough the structure

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is fine to add the test here, but I wonder if we need to include CRD this way 🤔

@coleenquadros, I think the Environment struct has a CRDs field that can take in a Go CustomResourceDefinition struct. I wonder if we can import that struct from Prometheus Operator somehow? That way we don't need to maintain the CRD in-tree

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintaining the CRD in-tree is what I have the issue with. Ideally we would keep the integration tests as light as possible and enable running locally with out any network for example, as they are now.

Why I am suggesting adding this to e2e now instead is because we already take care of installing Prom operator there from my understanding and we can push much of the slower e2e work to ci. Thoughts?

Copy link
Member

@saswatamcode saswatamcode Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup that works too, essentially skip testing this in integration.

If CRD isn't really available in Go way, makes sense to skip it. But I think Prometheus Operator may have some method that spits out Go CRD struct. We can then just use that?

},

ErrorIfCRDPathMissing: true,

// The BinaryAssetsDirectory is only required if you want to run the tests directly
Expand All @@ -85,6 +91,9 @@ var _ = BeforeSuite(func() {
err = monitoringthanosiov1alpha1.AddToScheme(scheme.Scheme)
Expect(err).NotTo(HaveOccurred())

err = monitoringv1.AddToScheme(scheme.Scheme)
Expect(err).NotTo(HaveOccurred())

//+kubebuilder:scaffold:scheme

k8sManager, err := ctrl.NewManager(cfg, ctrl.Options{
Expand Down
11 changes: 11 additions & 0 deletions internal/controller/thanosquery_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import (
"fmt"

"github.com/go-logr/logr"
monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1"
"github.com/prometheus/client_golang/prometheus"

monitoringthanosiov1alpha1 "github.com/thanos-community/thanos-operator/api/v1alpha1"
Expand Down Expand Up @@ -81,6 +82,7 @@ func NewThanosQueryReconciler(logger logr.Logger, client client.Client, scheme *
//+kubebuilder:rbac:groups=monitoring.thanos.io,resources=thanosqueries/finalizers,verbs=update
//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups="",resources=services;configmaps;serviceaccounts,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=monitoring.coreos.com,resources=servicemonitors,verbs=get;list;watch;create;update;patch;delete

// Reconcile is part of the main kubernetes reconciliation loop which aims to
// move the current state of the cluster closer to the desired state.
Expand Down Expand Up @@ -168,6 +170,13 @@ func (r *ThanosQueryReconciler) syncResources(ctx context.Context, query monitor
"resource", obj.GetName(), "namespace", obj.GetNamespace(),
)
}
if query.Spec.EnableSelfMonitor != nil && !*query.Spec.EnableSelfMonitor {
err := manifests.DeleteServiceMonitor(ctx, r.Client, query.Name, query.Namespace)
if err != nil {
philipgough marked this conversation as resolved.
Show resolved Hide resolved
r.logger.Error(err, "failed to delete ServiceMonitor")
errCount++
}
}

if errCount > 0 {
r.ControllerBaseMetrics.ClientErrorsTotal.WithLabelValues(manifestquery.Name).Add(float64(errCount))
Expand All @@ -187,6 +196,7 @@ func (r *ThanosQueryReconciler) buildQuerier(ctx context.Context, query monitori
LogLevel: query.Spec.LogLevel,
LogFormat: query.Spec.LogFormat,
ResourceRequirements: query.Spec.ResourceRequirements,
EnableServiceMonitor: *query.Spec.EnableSelfMonitor,
}.ApplyDefaults()

endpoints, err := r.getStoreAPIServiceEndpoints(ctx, query)
Expand Down Expand Up @@ -319,6 +329,7 @@ func (r *ThanosQueryReconciler) SetupWithManager(mgr ctrl.Manager) error {
Owns(&corev1.ServiceAccount{}).
Owns(&corev1.Service{}).
Owns(&appsv1.Deployment{}).
Owns(&monitoringv1.ServiceMonitor{}).
Watches(
&corev1.Service{},
r.enqueueForService(),
Expand Down
19 changes: 16 additions & 3 deletions internal/controller/thanosquery_controller_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,17 @@ import (
"fmt"
"time"

"github.com/thanos-community/thanos-operator/internal/pkg/manifests"
manifestquery "github.com/thanos-community/thanos-operator/internal/pkg/manifests/query"
appsv1 "k8s.io/api/apps/v1"

. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"

monitoringthanosiov1alpha1 "github.com/thanos-community/thanos-operator/api/v1alpha1"
"github.com/thanos-community/thanos-operator/internal/pkg/manifests"
manifestquery "github.com/thanos-community/thanos-operator/internal/pkg/manifests/query"
"github.com/thanos-community/thanos-operator/internal/pkg/manifests/receive"
"github.com/thanos-community/thanos-operator/test/utils"

appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
Expand Down Expand Up @@ -245,6 +246,18 @@ var _ = Describe("ThanosQuery Controller", Ordered, func() {
}, time.Second*30, time.Second*10).Should(Succeed())
})

By("removing service monitor when disabled", func() {
Expect(utils.VerifyServiceMonitor(k8sClient, resourceName, ns)).To(BeTrue())

enableSelfMonitor := false
resource.Spec.EnableSelfMonitor = &enableSelfMonitor
Expect(k8sClient.Update(context.Background(), resource)).Should(Succeed())

Eventually(func() bool {
return utils.VerifyServiceMonitorDeleted(k8sClient, resourceName, ns)
}, time.Minute*1, time.Second*10).Should(BeTrue())
})

By("checking paused state", func() {
isPaused := true
resource.Spec.Paused = &isPaused
Expand Down
Loading
Loading