developer_guide.md

Developer Guide

Project overview

Kubernetes Operator for Apache Flink is built on top of the Kubernetes controller-runtime library. The project structure and boilerplate files are generated with Kubebuilder. Knowledge of controller-runtime and Kubebuilder is required to understand this project.

The Flink custom resource is defined in Go struct FlinkCluster, then Kubebuild generates related Go files and YAML files, e.g. flinkclusters.yaml. The custom logic for reconciling a Flink custom resource is inside of the controllers directory, e.g., flinkcluster_controller.go.

Dockerfile defines the steps of building the Flink Operator image.

Makefile includes various actions you can take to generate code, build the Flink Operator binary, run unit tests, build and push docker image, deploy the Flink Operator to a Kubernetes cluster.

You might want to watch this tech talk (video, slides) to get a high-level overview of the project.

Dependencies

The following dependencies are required to build the Flink Operator binary and run unit tests:

• Go v1.14+
• Kubebuilder v3+

But you don't have to install them on your local machine, the Makefile will take care of those dependencies.

To create the Flink Operator Docker image and deploy it to a Kubernetes cluster, the following dependencies are required on your local machine:

• Docker
• kubectl v1.15+

Local build and test

To build the Flink Operator binary and run unit tests, run:

make test

Build and push the operator image

Build a Docker image for the Flink Operator and then push it to an image registry with

make docker-push IMG=<tag>

For example, if you are using Google Container Registry, follow the instructions here to set it up, then build the image and push it to GCR with:

PROJECT=<gcp-project>
IMAGE_TAG=gcr.io/${PROJECT}/flink-operator:latest
make docker-push IMG=${IMAGE_TAG}

Deploy the operator and run jobs

Now you can follow the User Guide to deploy the operator and run jobs.