Skip to content

Commit

Permalink
feat: repo setup and install grafana in e4ca1d (#36)
Browse files Browse the repository at this point in the history
* fix: repo setup and install grafana in e4ca1d

* feat: ignore helm charts from githooks

* feat: updated hooks

* feat: added code owners
  • Loading branch information
NithinKuruba authored Oct 19, 2023
1 parent d43165f commit 49fa76a
Show file tree
Hide file tree
Showing 10 changed files with 272 additions and 34 deletions.
98 changes: 98 additions & 0 deletions .gitlint
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# This file controls the behaviour of the `gitlint` tool and by extension the commit-msg pre-commit hook.
# @see http://jorisroovers.com/gitlint/configuration/

# All these sections are optional, edit this file as you like.
[general]
# Ignore certain rules, you can reference them by their id or by their full name
# ignore=title-trailing-punctuation, T3
# allow missing body message
ignore=B6

# verbosity should be a value between 1 and 3, the commandline -v flags take precedence over this
# verbosity = 2

# By default gitlint will ignore merge commits. Set to 'false' to disable.
# ignore-merge-commits=true

# By default gitlint will ignore fixup commits. Set to 'false' to disable.
# ignore-fixup-commits=true

# By default gitlint will ignore squash commits. Set to 'false' to disable.
# ignore-squash-commits=true

# Ignore any data send to gitlint via stdin
# ignore-stdin=true

# Enable debug mode (prints more output). Disabled by default.
# debug=true

# Enable community contributed rules
# See http://jorisroovers.github.io/gitlint/contrib_rules for details
contrib=contrib-title-conventional-commits

# Set the extra-path where gitlint will search for user defined rules
# See http://jorisroovers.github.io/gitlint/user_defined_rules for details
# extra-path=examples/

# [title-max-length]
# line-length=80

# [title-must-not-contain-word]
# Comma-separated list of words that should not occur in the title. Matching is case
# insensitive. It's fine if the keyword occurs as part of a larger word (so "WIPING"
# will not cause a violation, but "WIP: my title" will.
# words=wip

# [title-match-regex]
# python like regex (https://docs.python.org/2/library/re.html) that the
# commit-msg title must be matched to.
# Note that the regex can contradict with other rules if not used correctly
# (e.g. title-must-not-contain-word).
# regex=^US[0-9]*

[B1]
# B1 = body-max-line-length
line-length=160

# [body-min-length]
# min-length=5

# [body-is-missing]
# Whether to ignore this rule on merge commits (which typically only have a title)
# default = True
# ignore-merge-commits=false

# [body-changed-file-mention]
# List of files that need to be explicitly mentioned in the body when they are changed
# This is useful for when developers often erroneously edit certain files or git submodules.
# By specifying this rule, developers can only change the file when they explicitly reference
# it in the commit message.
# files=gitlint/rules.py,README.md

# [author-valid-email]
# python like regex (https://docs.python.org/2/library/re.html) that the
# commit author email address should be matched to
# For example, use the following regex if you only want to allow email addresses from foo.com
# regex=[^@][email protected]

# [ignore-by-title]
# Ignore certain rules for commits of which the title matches a regex
# E.g. Match commit titles that start with "Release"
# regex=^Release(.*)
#
# Ignore certain rules, you can reference them by their id or by their full name
# Use 'all' to ignore all rules
# ignore=T1,body-min-length

# [ignore-by-body]
# Ignore certain rules for commits of which the body has a line that matches a regex
# E.g. Match bodies that have a line that that contain "release"
# regex=(.*)release(.*)
#
# Ignore certain rules, you can reference them by their id or by their full name
# Use 'all' to ignore all rules
# ignore=T1,body-min-length

[contrib-title-conventional-commits]
# Specify allowed commit types. For details see: https://www.conventionalcommits.org/
types = fix,feat,chore,docs,style,refactor,perf,test,revert,ci,build
4 changes: 2 additions & 2 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
nodejs 16.14.0
python 3.11.0
postgres 14.5
postgres 14.1
golang 1.19.4
helm 3.10.3
helm 3.10.2
5 changes: 5 additions & 0 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# These owners will be the default owners for everything in

# the repo. Unless a later match takes precedence.

- @NithinKuruba @thegentlemanphysicist @arcshiftsolutions @jlangy
9 changes: 9 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.PHONY: local-setup
local-setup:
cat .tool-versions | cut -f 1 -d ' ' | xargs -n 1 asdf plugin-add || true
asdf plugin-update --all
asdf install
asdf reshim
pip install -r requirements.txt
pre-commit install
gitlint install-hook
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

SSO Keycloak dashboard services provide the ability to monitor real-time statistical data and event logs.

## Local Development Environment

- Install asdf
- Run `make local-setup` to install necessary tooling

## Benefits

1. De-coupling the auditing service from the authentication service (Keycloak) and reducing the amount of Keycloak SQL transactions and DB data storage; gives better maintainability of the Keycloak instances.
Expand Down
7 changes: 1 addition & 6 deletions helm/grafana/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,6 @@ NAMESPACE=
SSO_CLIENT_ID=
SSO_CLIENT_SECRET=

AGGREGATION_DB_SECRET := sso-aggregator-patroni-appusers
AGGREGATION_DB_USER := $(shell kubectl get secret -n "${NAMESPACE}" "${AGGREGATION_DB_SECRET}" -o jsonpath="{.data.username-appuser1}" | base64 --decode)
AGGREGATION_DB_PASS := $(shell kubectl get secret -n "${NAMESPACE}" "${AGGREGATION_DB_SECRET}" -o jsonpath="{.data.password-appuser1}" | base64 --decode)
NAME := sso-grafana

ifndef NAMESPACE
Expand All @@ -15,9 +12,7 @@ endif
define arguments
"${NAME}" . -n "${NAMESPACE}" -f values.yaml -f "values-${NAMESPACE}.yaml" \
--set grafana."grafana\.ini"."auth\.generic_oauth".client_id=${SSO_CLIENT_ID} \
--set grafana."grafana\.ini"."auth\.generic_oauth".client_secret=${SSO_CLIENT_SECRET} \
--set grafana.datasources."datasources\.yaml".datasources[1].user=${AGGREGATION_DB_USER} \
--set grafana.datasources."datasources\.yaml".datasources[1].secureJsonData.password=${AGGREGATION_DB_PASS}
--set grafana."grafana\.ini"."auth\.generic_oauth".client_secret=${SSO_CLIENT_SECRET}
endef

.PHONY: helm-dep
Expand Down
37 changes: 36 additions & 1 deletion helm/grafana/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,41 @@ A Helm chart for deploying [Grafana dashboard](https://github.com/grafana/helm-c

## Local deployment via Helm chart

### Pre-Requisites

#### Setup Network Policies

- Below network policy has to be added to all the namespaces, where grafana requires to access the data source

```yaml
# Update $LICENSE_PLATE (ex.: e4ca1d)

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: sso-dev-sandbox-gold-grafana-access
namespace: $LICENSE_PLATE-(dev/test/prod)
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: sso-patroni
ingress:
- from:
- namespaceSelector:
matchLabels:
environment: tools
name: $LICENSE_PLATE
- podSelector:
matchLabels:
app.kubernetes.io/name: sso-grafana
policyTypes:
- Ingress
```
#### Update Helm Values
- Update data source username, password and database names under `values-$LICENSE_PLATE.yml` in place of `<please-replace-me>`

### Installing/Upgrading the Chart

```sh
Expand All @@ -24,7 +59,7 @@ make uninstall NAMESPACE=<namespace>

once the deployment is completed, please find the Grafana admin credentials in OCP secrets below:

- `dev`: https://console.apps.gold.devops.gov.bc.ca/k8s/ns/c6af30-prod/secrets/sso-grafana
- `dev`: https://console.apps.gold.devops.gov.bc.ca/k8s/ns/e4ca1d-tools/secrets/sso-grafana
- `prod`: https://console.apps.gold.devops.gov.bc.ca/k8s/ns/eb75ad-prod/secrets/sso-grafana

## Configuration
Expand Down
89 changes: 89 additions & 0 deletions helm/grafana/values-e4ca1d-tools.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
grafana:
replicas: 1
persistence:
size: 1Gi

grafana.ini:
server:
domain: sso-grafana-sandbox.apps.gold.devops.gov.bc.ca
root_url: https://sso-grafana-sandbox.apps.gold.devops.gov.bc.ca
auth.generic_oauth:
auth_url: https://dev.sandbox.loginproxy.gov.bc.ca/auth/realms/standard/protocol/openid-connect/auth
token_url: https://dev.sandbox.loginproxy.gov.bc.ca/auth/realms/standard/protocol/openid-connect/token

datasources:
datasources.yaml:
apiVersion: 1
datasources:
- name: SSO Logs
type: postgres
access: proxy
orgId: 1
url: kc-cron-patroni-readonly:5432
user: <please-replace-me>
database: <please-replace-me>
basicAuth: false
secureJsonData:
password: <please-replace-me>
jsonData:
sslmode: disable
maxOpenConns:
maxIdleConns: 2
connMaxLifetime: 14400
postgresVersion: 1400
timescaledb: false
- name: SSO Keycloak Dev
type: postgres
access: proxy
orgId: 1
url: sso-patroni-readonly.e4ca1d-dev.svc.cluster.local:5432
user: <please-replace-me>
database: <please-replace-me>
basicAuth: false
secureJsonData:
password: <please-replace-me>
jsonData:
sslmode: disable
maxOpenConns:
maxIdleConns: 2
connMaxLifetime: 14400
postgresVersion: 1400
timescaledb: false
- name: SSO Keycloak Test
type: postgres
access: proxy
orgId: 1
url: sso-patroni-readonly.e4ca1d-test.svc.cluster.local:5432
user: <please-replace-me>
database: <please-replace-me>
basicAuth: false
secureJsonData:
password: <please-replace-me>
jsonData:
sslmode: disable
maxOpenConns:
maxIdleConns: 2
connMaxLifetime: 14400
postgresVersion: 1400
timescaledb: false
- name: SSO Keycloak Prod
type: postgres
access: proxy
orgId: 1
url: sso-patroni-readonly.e4ca1d-prod.svc.cluster.local:5432
user: <please-replace-me>
database: <please-replace-me>
basicAuth: false
secureJsonData:
password: <please-replace-me>
jsonData:
sslmode: disable
maxOpenConns:
maxIdleConns: 2
connMaxLifetime: 14400
postgresVersion: 1400
timescaledb: false

route:
enabled: true
host: sso-grafana-sandbox.apps.gold.devops.gov.bc.ca
51 changes: 26 additions & 25 deletions helm/grafana/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ grafana:

image:
repository: grafana/grafana
tag: "9.3.2"
tag: '9.3.2'
pullPolicy: IfNotPresent
pullSecrets: []

Expand Down Expand Up @@ -75,36 +75,37 @@ grafana:
tls_skip_verify_insecure: true
use_pkce: true
login_attribute_path: preferred_username
role_attribute_path: contains(client_roles[*], 'grafanaadmin') && 'GrafanaAdmin' || contains(client_roles[*], 'admin') && 'Admin' || contains(client_roles[*], 'editor') && 'Editor' || 'Viewer'

# see https://github.com/kubeflow/community/blob/master/devstats/config/grafana/provisioning/datasources/datasource.yaml
datasources:
datasources.yaml:
apiVersion: 1
datasources:
- name: SSO Loki
type: loki
access: proxy
orgId: 1
url: http://sso-loki-gateway
basicAuth: false
isDefault: true
- name: SSO Postgres
type: postgres
access: proxy
orgId: 1
url: sso-aggregator-patroni-readonly:5432
user: <please-replace-me>
database: aggregation
basicAuth: false
secureJsonData:
password: <please-replace-me>
jsonData:
sslmode: disable
maxOpenConns:
maxIdleConns: 2
connMaxLifetime: 14400
postgresVersion: 1400
timescaledb: false
- name: SSO Loki
type: loki
access: proxy
orgId: 1
url: http://sso-loki-gateway
basicAuth: false
isDefault: true
- name: SSO Postgres
type: postgres
access: proxy
orgId: 1
url: sso-aggregator-patroni-readonly:5432
user: <please-replace-me>
database: aggregation
basicAuth: false
secureJsonData:
password: <please-replace-me>
jsonData:
sslmode: disable
maxOpenConns:
maxIdleConns: 2
connMaxLifetime: 14400
postgresVersion: 1400
timescaledb: false

route:
enabled: true
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
pre-commit==2.9.3
gitlint==0.15.1

0 comments on commit 49fa76a

Please sign in to comment.