Skip to content

Commit

Permalink
Initial commit of Filebeat chart
Browse files Browse the repository at this point in the history
  • Loading branch information
@tylerjl
tylerjl committed Apr 26, 2019
1 parent 2a231f2 commit b367fa5
Show file tree
Hide file tree
Showing 17 changed files with 478 additions and 0 deletions.
1 change: 1 addition & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Please look in the chart directories for the documentation for each chart. These

* [Elasticsearch](./elasticsearch/README.md) - [docker image docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html)
* [Kibana](./kibana/README.md) - [docker image docs](https://www.elastic.co/guide/en/kibana/current/docker.html)
* [Filebeat](./filebeat/README.md) - [docker image docs](https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html)

## Kubernetes versions

Expand Down
2 changes: 2 additions & 0 deletions filebeat/.helmignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
tests/
.pytest_cache/
11 changes: 11 additions & 0 deletions filebeat/Chart.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
description: Filebeat
home: https://github.com/elastic/helm-charts
maintainers:
- email: [email protected]
name: Elastic
name: filebeat
version: 7.0.0-alpha1
appVersion: 7.0.0
sources:
- https://github.com/elastic/beats
icon: https://helm.elastic.co/icons/filebeat.png
1 change: 1 addition & 0 deletions filebeat/Makefile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../helpers/common.mk
110 changes: 110 additions & 0 deletions filebeat/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Filebeat Helm Chart

This functionality is in alpha status and may be changed or removed completely in a future release. Elastic will take a best effort approach to fix any issues, but alpha features are not subject to the support SLA of official GA features.

This helm chart is a lightweight way to configure and run our official [Filebeat docker image](https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html).

## Requirements

* Kubernetes >= 1.8
* [Helm](https://helm.sh/) >= 2.8.0

## Installing

* Add the elastic helm charts repo
```
helm repo add elastic https://helm.elastic.co
```
* Install it
```
helm install --name filebeat elastic/filebeat --version 7.0.0-alpha1
```

## Compatibility

This chart is tested with the latest supported versions. The currently tested versions are:

| 5.x | 6.x | 7.x |
| ------ | ----- | ----- |
| 5.6.16 | 6.7.1 | 7.0.0 |

Examples of installing older major versions can be found in the [examples](./examples) directory.

While only the latest releases are tested, it is possible to easily install old or new releases by overriding the `imageTag`. To install version `7.0.0` of Filebeat it would look like this:

```
helm install --name filebeat elastic/filebeat --version 7.0.0-alpha1 --set imageTag=7.0.0
```


## Configuration
| Parameter | Description | Default |
| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `filebeatConfig` | Allows you to add any config files in `/usr/share/filebeat` such as `filebeat.yml`. See [values.yaml](./values.yaml) for an example of the formatting. | `{}` |
| `extraEnvs` | Extra [environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/#using-environment-variables-inside-of-your-config) which will be appended to the `env:` definition for the container | `[]` |
| `extraVolumeMounts` | Any extra volumes mounts to define for the Filebeat container | `[]` |
| `extraVolumes` | Any extra volumes to define for the pod | `[]` |
| `hostPathRoot` | Fully-qualified [hostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath) that will be used to persist Filebeat registry data | `/var/lib` |
| `image` | The Filebeat docker image | `docker.elastic.co/beats/filebeat` |
| `imageTag` | The Filebeat docker image tag | `7.0.0` |
| `imagePullPolicy` | The Kubernetes [imagePullPolicy](https://kubernetes.io/docs/concepts/containers/images/#updating-images) value | `IfNotPresent` |
| `imagePullSecrets` | Configuration for [imagePullSecrets](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-pod-that-uses-your-secret) so that you can use a private registry for your image | `[]` |
| `managedServiceAccount` | Whether the `serviceAccount` should be managed by this helm chart. Set this to `false` in order to manage your own service account and related roles. | `true` |
| `podAnnotations` | Configurable [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) applied to all Filebeat pods | `{}` |
| `podSecurityContext` | Configurable [podSecurityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for Filebeat pod execution environment | `fsGroup: 1000`<br>`runAsUser: 0`<br>`privileged: false` |
| `livenessProbe` | Parameters to pass to [liveness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/) checks for values such as timeouts and thresholds. | `failureThreshold: 3`<br>`initialDelaySeconds: 10`<br>`periodSeconds: 10`<br>`successThreshold: 3`<br>`timeoutSeconds: 5` |
| `resources` | Allows you to set the [resources](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for the `DaemonSet` | `requests.cpu: 100m`<br>`requests.memory: 100Mi`<br>`limits.cpu: 1000m`<br>`limits.memory: 200Mi` |
| `serviceAccount` | The [serviceAccount](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) that Filebeat will use during execution | `filebeat` |
| `secretMounts` | Allows you easily mount a secret as a file inside the `DaemonSet`. Useful for mounting certificates and other secrets. See [values.yaml](./values.yaml) for an example | `[]` |
| `terminationGracePeriod` | Termination period (in seconds) to wait before killing Filebeat pod process on pod shutdown | `30` |
| `tolerations` | Configurable [tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) | `[]` |
| `updateStrategy` | The [updateStrategy](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#updating-statefulsets) for the `DaemonSet`. By default Kubernetes will kill and recreate pods on updates. Setting this to `OnDelete` will require that pods be deleted manually. | `RollingUpdate` |

## Examples

In [examples/](./examples) you will find some example configurations. These examples are used for the automated testing of this helm chart.

### Default

* Deploy the [default Elasticsearch helm chart](../elasticsearch/README.md#default)
* Deploy Filebeat with the default values
```
cd examples/default
make
```
* You can now setup a port forward for Elasticsearch to observe Filebeat indices
```
kubectl port-forward svc/elasticsearch-master 9200
curl localhost:9200/_cat/indices
```

## Testing

This chart uses [pytest](https://docs.pytest.org/en/latest/) to test the templating logic. The dependencies for testing can be installed from the [`requirements.txt`](../requirements.txt) in the parent directory.

```
pip install -r ../requirements.txt
make pytest
```

You can also use `helm template` to look at the YAML being generated

```
make template
```

It is possible to run all of the tests and linting inside of a docker container

```
make test
```

## Integration Testing

Integration tests are run using [goss](https://github.com/aelsabbahy/goss/blob/master/docs/manual.md) which is a serverspec like tool written in golang. See [goss.yaml](examples/default/test/goss.yaml) for an example of what the tests look like.

To run the goss tests against the default example:
```
cd examples/default
make goss
```
13 changes: 13 additions & 0 deletions filebeat/examples/default/Makefile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
default: test

include ../../../helpers/examples.mk

RELEASE := helm-filebeat-default

install:
helm upgrade --wait --timeout=600 --values values.yaml --install $(RELEASE) ../../

test: install goss

purge:
helm del --purge $(RELEASE)
15 changes: 15 additions & 0 deletions filebeat/examples/default/test/goss.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
port:
tcp:5066:
listening: true
ip:
- '127.0.0.1'

mount:
/usr/share/filebeat/data:
exists: true

user:
filebeat:
exists: true
uid: 1000
gid: 1000
17 changes: 17 additions & 0 deletions filebeat/examples/default/values.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
filebeatConfig:
filebeat.yml: |
filebeat.inputs:
- type: docker
containers.ids:
- '*'
processors:
- add_kubernetes_metadata:
in_cluster: true
output.elasticsearch:
hosts: '${ELASTICSEARCH_HOSTS:elasticsearch:9200}'
extraEnvs:
- name: ELASTICSEARCH_HOSTS
value: elasticsearch-master:9200
4 changes: 4 additions & 0 deletions filebeat/templates/NOTES.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
1. Watch all cluster members come up.
$ kubectl get pods --namespace={{ .Release.Namespace }} -l app={{ template "fullname" . }} -w
2. Test cluster health using Helm test.
$ helm test {{ .Release.Name }}
16 changes: 16 additions & 0 deletions filebeat/templates/_helpers.tpl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
*/}}
{{- define "fullname" -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
21 changes: 21 additions & 0 deletions filebeat/templates/clusterrole.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{{- if .Values.managedServiceAccount }}
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: {{ .Values.serviceAccount }}-cluster-role
labels:
app: "{{ template "fullname" . }}"
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
heritage: {{ .Release.Service | quote }}
release: {{ .Release.Name | quote }}
rules:
- apiGroups:
- ""
resources:
- namespaces
- pods
verbs:
- get
- list
- watch
{{- end -}}
19 changes: 19 additions & 0 deletions filebeat/templates/clusterrolebinding.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{{- if .Values.managedServiceAccount }}
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: {{ .Values.serviceAccount }}-cluster-role-binding
labels:
app: "{{ template "fullname" . }}"
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
heritage: {{ .Release.Service | quote }}
release: {{ .Release.Name | quote }}
roleRef:
kind: ClusterRole
name: {{ .Values.serviceAccount }}-cluster-role
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: {{ .Values.serviceAccount }}
namespace: {{ .Release.Namespace }}
{{- end -}}
17 changes: 17 additions & 0 deletions filebeat/templates/configmap.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{{- if .Values.filebeatConfig }}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "name" . }}-config
labels:
app: "{{ template "name" . }}"
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
heritage: {{ .Release.Service | quote }}
release: {{ .Release.Name | quote }}
data:
{{- range $path, $config := .Values.filebeatConfig }}
{{ $path }}: |
{{ $config | indent 4 -}}
{{- end -}}
{{- end -}}
Loading

0 comments on commit b367fa5

Please sign in to comment.