Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for alert notification templates #531

Merged
merged 8 commits into from
Jan 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 23 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,30 @@
# Grizzly: Manage your Observability Systems

A utility for managing various observability resources with Jsonnet. Currently supported
are:
Grafana Grizzly is a command line tool that allows you to manage your observability resources as code.

* Grafana dashboards/dashboard folders
* Grafana datasources
* Grafana Cloud Prometheus recording rules/alerts
* Grafana Synthetic Monitoring checks
Now you can define your dashboards, alerting and recording rules,… all from within your codebase.

See [docs](https://grafana.github.io/grizzly) for usage details.
You can build Grizzly into your continuous deployment pipelines, meaning whenever you deploy your application, your observability is updated too.

Currently supported are:

* [Grafana](https://grafana.github.io/grizzly/grafana/)
* [Alerting contact points](https://grafana.github.io/grizzly/grafana/#contact-points)
* [Alerting notification templates](https://grafana.github.io/grizzly/grafana/#notification-templates)
* [Alerting notification policy](https://grafana.github.io/grizzly/grafana/#notification-policy)
* [Alert rule groups](https://grafana.github.io/grizzly/grafana/#alertrulegroup)
* [Dashboards](https://grafana.github.io/grizzly/grafana/#dashboards)
* [Dashboard folders](https://grafana.github.io/grizzly/grafana/#folders)
* [Datasources](https://grafana.github.io/grizzly/grafana/#datasources)
* [Library elements](https://grafana.github.io/grizzly/grafana/#library-elements)
* [Prometheus](https://grafana.github.io/grizzly/prometheus/)
* [Alerts](https://grafana.github.io/grizzly/prometheus/#prometheus-alerts)
* [Recording rules](https://grafana.github.io/grizzly/prometheus/#prometheus-recording-rules)
* [Grafana Synthetic Monitoring](https://grafana.github.io/grizzly/synthetic-monitoring/)
* [Monitoring Checks](https://grafana.github.io/grizzly/synthetic-monitoring/#grafana-cloud-synthetic-monitoring-checks)

See [docs](https://grafana.github.io/grizzly) for more details.

## Contributing

See our [contributing guide](CONTRIBUTING.md).
See our [contributing guide](CONTRIBUTING.md).
65 changes: 52 additions & 13 deletions docs/content/grafana.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,18 +219,57 @@ name `global`:
apiVersion: grizzly.grafana.com/v1alpha1
kind: AlertNotificationPolicy
metadata:
name: global
name: global
spec:
group_by:
- grafana_folder
- alertname
receiver: grafana-default-email
routes:
- group_by:
- region
object_matchers:
- - foo
- =
- bar
receiver: grafana-oncall
group_by:
- grafana_folder
- alertname
receiver: grafana-default-email
routes:
- group_by:
- region
object_matchers:
- - foo
- =
- bar
receiver: grafana-oncall
```

## Notification Templates

To notification templates, use the following structure:

```yaml
apiVersion: grizzly.grafana.com/v1alpha1
kind: AlertNotificationTemplate
metadata:
name: standard-template
spec:
name: standard-template
template: |-
{{ define "default.title.copy" }}
[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ if gt (.Alerts.Resolved | len) 0 }}, RESOLVED:{{ .Alerts.Resolved | len }}{{ end }}{{ end }}] {{ .GroupLabels.SortedPairs.Values | join " " }} {{ if gt (len .CommonLabels) (len .GroupLabels) }}({{ with .CommonLabels.Remove .GroupLabels.Names }}{{ .Values | join " " }}{{ end }}){{ end }}
{{ end }}

{{ define "default.message.copy" }}{{ if gt (len .Alerts.Firing) 0 }}**Firing**
{{ template "__text_alert_list.copy" .Alerts.Firing }}{{ if gt (len .Alerts.Resolved) 0 }}

{{ end }}{{ end }}{{ if gt (len .Alerts.Resolved) 0 }}**Resolved**
{{ template "__text_alert_list.copy" .Alerts.Resolved }}{{ end }}{{ end }}

{{ define "__text_alert_list.copy" }}{{ range . }}
Value: {{ template "__text_values_list.copy" . }}
Labels:
{{ range .Labels.SortedPairs }} - {{ .Name }} = {{ .Value }}
{{ end }}Annotations:
{{ range .Annotations.SortedPairs }} - {{ .Name }} = {{ .Value }}
{{ end }}{{ if gt (len .GeneratorURL) 0 }}Source: {{ .GeneratorURL }}
{{ end }}{{ if gt (len .SilenceURL) 0 }}Silence: {{ .SilenceURL }}
{{ end }}{{ if gt (len .DashboardURL) 0 }}Dashboard: {{ .DashboardURL }}
{{ end }}{{ if gt (len .PanelURL) 0 }}Panel: {{ .PanelURL }}
{{ end }}{{ end }}{{ end }}

{{ define "__text_values_list.copy" }}{{ if len .Values }}{{ $first := true }}{{ range $refID, $value := .Values -}}
{{ if $first }}{{ $first = false }}{{ else }}, {{ end }}{{ $refID }}={{ $value }}{{ end -}}
{{ else }}[no value]{{ end }}{{ end }}
```
92 changes: 92 additions & 0 deletions integration/alertnotificationtemplate_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
package integration_test

import (
"testing"
)

func TestAlertNotificationTemplates(t *testing.T) {
dir := "testdata/alertnotificationtemplates"
setupContexts(t, dir)

t.Run("Get template - not found", func(t *testing.T) {
runTest(t, GrizzlyTest{
TestDir: dir,
RunOnContexts: allContexts,
Commands: []Command{
{
Command: "get AlertNotificationTemplate.dummy",
ExpectedCode: 1,
},
},
})
})

t.Run("Apply template", func(t *testing.T) {
runTest(t, GrizzlyTest{
TestDir: dir,
RunOnContexts: allContexts,
Commands: []Command{
{
Command: "apply standard-template.yaml",
ExpectedCode: 0,
},
},
})
})

t.Run("Apply same template", func(t *testing.T) {
runTest(t, GrizzlyTest{
TestDir: dir,
RunOnContexts: allContexts,
Commands: []Command{
{
Command: "apply standard-template.yaml",
ExpectedCode: 0,
ExpectedOutputContains: "AlertNotificationTemplate.standard-template unchanged\n",
},
},
})
})

t.Run("Get applied template", func(t *testing.T) {
runTest(t, GrizzlyTest{
TestDir: dir,
RunOnContexts: allContexts,
Commands: []Command{
{
Command: "get AlertNotificationTemplate.standard-template",
ExpectedCode: 0,
ExpectedOutputFile: "standard-template.yaml",
},
},
})
})

t.Run("Get remote templates list", func(t *testing.T) {
runTest(t, GrizzlyTest{
TestDir: dir,
RunOnContexts: allContexts,
Commands: []Command{
{
Command: "list -r -t AlertNotificationTemplate",
ExpectedCode: 0,
ExpectedOutputFile: "list.txt",
},
},
})
})

t.Run("Diff template with no differences", func(t *testing.T) {
runTest(t, GrizzlyTest{
TestDir: dir,
RunOnContexts: allContexts,
Commands: []Command{
{
Command: "diff standard-template.yaml",
ExpectedCode: 0,
ExpectedOutputContains: "AlertNotificationTemplate.standard-template no differences",
},
},
})
})
}
2 changes: 2 additions & 0 deletions integration/testdata/alertnotificationtemplates/list.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
API VERSION KIND UID
grizzly.grafana.com/v1alpha1 AlertNotificationTemplate standard-template
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
apiVersion: grizzly.grafana.com/v1alpha1
kind: AlertNotificationTemplate
metadata:
name: standard-template
spec:
name: standard-template
template: |-
{{ define "default.title.copy" }}
[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ if gt (.Alerts.Resolved | len) 0 }}, RESOLVED:{{ .Alerts.Resolved | len }}{{ end }}{{ end }}] {{ .GroupLabels.SortedPairs.Values | join " " }} {{ if gt (len .CommonLabels) (len .GroupLabels) }}({{ with .CommonLabels.Remove .GroupLabels.Names }}{{ .Values | join " " }}{{ end }}){{ end }}
{{ end }}

{{ define "default.message.copy" }}{{ if gt (len .Alerts.Firing) 0 }}**Firing**
{{ template "__text_alert_list.copy" .Alerts.Firing }}{{ if gt (len .Alerts.Resolved) 0 }}

{{ end }}{{ end }}{{ if gt (len .Alerts.Resolved) 0 }}**Resolved**
{{ template "__text_alert_list.copy" .Alerts.Resolved }}{{ end }}{{ end }}

{{ define "__text_alert_list.copy" }}{{ range . }}
Value: {{ template "__text_values_list.copy" . }}
Labels:
{{ range .Labels.SortedPairs }} - {{ .Name }} = {{ .Value }}
{{ end }}Annotations:
{{ range .Annotations.SortedPairs }} - {{ .Name }} = {{ .Value }}
{{ end }}{{ if gt (len .GeneratorURL) 0 }}Source: {{ .GeneratorURL }}
{{ end }}{{ if gt (len .SilenceURL) 0 }}Silence: {{ .SilenceURL }}
{{ end }}{{ if gt (len .DashboardURL) 0 }}Dashboard: {{ .DashboardURL }}
{{ end }}{{ if gt (len .PanelURL) 0 }}Panel: {{ .PanelURL }}
{{ end }}{{ end }}{{ end }}

{{ define "__text_values_list.copy" }}{{ if len .Values }}{{ $first := true }}{{ range $refID, $value := .Values -}}
{{ if $first }}{{ $first = false }}{{ else }}, {{ end }}{{ $refID }}={{ $value }}{{ end -}}
{{ else }}[no value]{{ end }}{{ end }}
17 changes: 17 additions & 0 deletions internal/utils/arrays.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package utils

// Map creates a new list populated with the results of applying the `mapper`
// function to every element from `input`.
func Map[T any, O any](input []T, mapper func(T) O) []O {
K-Phoen marked this conversation as resolved.
Show resolved Hide resolved
if input == nil {
return nil
}

output := make([]O, len(input))

for i := range input {
output[i] = mapper(input[i])
}

return output
}
2 changes: 1 addition & 1 deletion pkg/grafana/contactpoint-handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ func (h *AlertContactPointHandler) Prepare(existing *grizzly.Resource, resource
func (h *AlertContactPointHandler) Validate(resource grizzly.Resource) error {
uid, exist := resource.GetSpecString("uid")
if exist && uid != resource.Name() {
return fmt.Errorf("uid '%s' and name '%s', don't match", uid, resource.Name())
return ErrUIDNameMismatch{UID: uid, Name: resource.Name()}
}
return nil
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/grafana/dashboard-handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ func (h *DashboardHandler) GetByUID(uid string) (*grizzly.Resource, error) {
func (h *DashboardHandler) GetRemote(resource grizzly.Resource) (*grizzly.Resource, error) {
uid, _ := resource.GetSpecString("uid")
if uid != resource.Name() {
return nil, fmt.Errorf("uid '%s' and name '%s', don't match", uid, resource.Name())
return nil, ErrUIDNameMismatch{UID: uid, Name: resource.Name()}
}
return h.getRemoteDashboard(resource.Name())
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/grafana/datasource-handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ func (h *DatasourceHandler) Validate(resource grizzly.Resource) error {
uid, exist := resource.GetSpecString("uid")
if exist {
if uid != resource.Name() {
return fmt.Errorf("uid '%s' and name '%s', don't match", uid, resource.Name())
return ErrUIDNameMismatch{UID: uid, Name: resource.Name()}
}
}
return nil
Expand Down
9 changes: 9 additions & 0 deletions pkg/grafana/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ import (
"strings"
)

type ErrUIDNameMismatch struct {
UID string
Name string
}

func (e ErrUIDNameMismatch) Error() string {
return fmt.Sprintf("uid '%s' and name '%s', don't match", e.UID, e.Name)
}

// ErrUidsMissing reports UIDs are missing for Dashboards
type ErrUidsMissing []string

Expand Down
2 changes: 1 addition & 1 deletion pkg/grafana/folder-handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ func (h *FolderHandler) Validate(resource grizzly.Resource) error {
uid, exist := resource.GetSpecString("uid")
if exist {
if uid != resource.Name() {
return fmt.Errorf("uid '%s' and name '%s', don't match", uid, resource.Name())
return ErrUIDNameMismatch{UID: uid, Name: resource.Name()}
}
}

Expand Down
2 changes: 1 addition & 1 deletion pkg/grafana/library-element-handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ func (h *LibraryElementHandler) Validate(resource grizzly.Resource) error {
uid, exist := resource.GetSpecString("uid")
if exist {
if uid != resource.Name() {
return fmt.Errorf("uid '%s' and name '%s', don't match", uid, resource.Name())
return ErrUIDNameMismatch{UID: uid, Name: resource.Name()}
}
}

Expand Down
Loading
Loading