Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/v3' into feature/rename-brokers
Browse files Browse the repository at this point in the history 
# Conflicts:
#	streams-bootstrap-cli/src/main/java/com/bakdata/kafka/KafkaApplication.java
  • Loading branch information
@philipp94831
philipp94831 committed Jul 24, 2024
2 parents 6d5c0d7 + 9b67bc7 commit 87e6bd4
Show file tree
Hide file tree
Showing 23 changed files with 95 additions and 95 deletions.
4 changes: 0 additions & 4 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,6 @@ The following configuration options are available:

- `--volatile-group-instance-id`: Whether the group instance id is volatile, i.e., it will change on a Streams shutdown.

- `--debug`: Configure logging to debug

Additionally, the following commands are available:

- `clean`: Reset the Kafka Streams application. Additionally, delete the consumer group and all output and intermediate
Expand Down Expand Up @@ -188,8 +186,6 @@ The following configuration options are available:

- `--extra-output-topics`: Additional named output topics (`String=String>[,<String=String>...]`)

- `--debug`: Configure logging to debug

Additionally, the following commands are available:

- `clean`: Delete all output topics associated with the Kafka Producer application.
Expand Down
6 changes: 1 addition & 5 deletions charts/producer-app-cleanup-job/templates/job.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ spec:
- name: ENV_PREFIX
value: {{ .Values.configurationEnvPrefix }}_
{{- range $key, $value := .Values.streams.config }}
- name: {{ printf "STREAMS_%s" $key | replace "." "_" | upper | quote }}
- name: {{ printf "KAFKA_%s" $key | replace "." "_" | upper | quote }}
value: {{ $value | quote }}
{{- end }}
{{- if hasKey .Values.streams "bootstrapServers" }}
Expand All @@ -72,10 +72,6 @@ spec:
- name: "{{ .Values.configurationEnvPrefix }}_SCHEMA_REGISTRY_URL"
value: {{ .Values.streams.schemaRegistryUrl | quote }}
{{- end }}
{{- if hasKey .Values "debug" }}
- name: "{{ .Values.configurationEnvPrefix }}_DEBUG"
value: {{ .Values.debug | quote }}
{{- end }}
{{- if hasKey .Values.streams "outputTopic" }}
- name: "{{ .Values.configurationEnvPrefix }}_OUTPUT_TOPIC"
value: {{ .Values.streams.outputTopic | quote }}
Expand Down
2 changes: 0 additions & 2 deletions charts/producer-app-cleanup-job/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,6 @@ streams:
commandLine: {}
# MY_CLI_PARAM: "foo-bar"

debug: false

env: {}
# MY_ENV_VARIABLE: foo-bar

Expand Down
19 changes: 9 additions & 10 deletions charts/producer-app/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,16 +56,15 @@ Alternatively, a YAML file that specifies the values for the parameters can be p

### Other

| Parameter | Description | Default |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `configurationEnvPrefix` | Prefix for environment variables to use that should be parsed as command line arguments. | `APP` |
| `commandLine` | Map of command line arguments passed to the producer app. | `{}` |
| `debug` | Configure logging to debug | `false` |
| `env` | Custom environment variables | `{}` |
| `secrets` | Custom secret environment variables. Prefix with `configurationEnvPrefix` in order to pass secrets to command line or prefix with `STREAMS_` to pass secrets to Kafka Streams configuration. E.g., `APP_MY_PARAM` would be passed as `--my-param` and `STREAMS_MAX_POLL_TIMEOUT_MS` would be translated to `max.poll.timeout.ms`. | `{}` |
| `secretRefs` | Inject existing secrets as environment variables. Map key is used as environment variable name. Value consists of secret `name` and `key`. | `{}` |
| `secretFilesRefs` | Mount existing secrets as volumes | `[]` |
| `files` | Map of files to mount for the app. File will be mounted as `$value.mountPath/$key`. `$value.content` denotes file content (recommended to be used with `--set-file`). | `{}` |
| Parameter | Description | Default |
|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|
| `configurationEnvPrefix` | Prefix for environment variables to use that should be parsed as command line arguments. | `APP` |
| `commandLine` | Map of command line arguments passed to the producer app. | `{}` |
| `env` | Custom environment variables | `{}` |
| `secrets` | Custom secret environment variables. Prefix with `configurationEnvPrefix` in order to pass secrets to command line or prefix with `KAFKA_` to pass secrets to Kafka Streams configuration. E.g., `APP_MY_PARAM` would be passed as `--my-param` and `KAFKA_MAX_POLL_TIMEOUT_MS` would be translated to `max.poll.timeout.ms`. | `{}` |
| `secretRefs` | Inject existing secrets as environment variables. Map key is used as environment variable name. Value consists of secret `name` and `key`. | `{}` |
| `secretFilesRefs` | Mount existing secrets as volumes | `[]` |
| `files` | Map of files to mount for the app. File will be mounted as `$value.mountPath/$key`. `$value.content` denotes file content (recommended to be used with `--set-file`). | `{}` |

### JVM

Expand Down
6 changes: 1 addition & 5 deletions charts/producer-app/templates/pod.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ spec:
- name: ENV_PREFIX
value: {{ .Values.configurationEnvPrefix }}_
{{- range $key, $value := .Values.streams.config }}
- name: {{ printf "STREAMS_%s" $key | replace "." "_" | upper | quote }}
- name: {{ printf "KAFKA_%s" $key | replace "." "_" | upper | quote }}
value: {{ $value | quote }}
{{- end }}
{{- if hasKey .Values.streams "bootstrapServers" }}
Expand All @@ -56,10 +56,6 @@ spec:
- name: "{{ .Values.configurationEnvPrefix }}_SCHEMA_REGISTRY_URL"
value: {{ .Values.streams.schemaRegistryUrl | quote }}
{{- end }}
{{- if hasKey .Values "debug" }}
- name: "{{ .Values.configurationEnvPrefix }}_DEBUG"
value: {{ .Values.debug | quote }}
{{- end }}
{{- if hasKey .Values.streams "outputTopic" }}
- name: "{{ .Values.configurationEnvPrefix }}_OUTPUT_TOPIC"
value: {{ .Values.streams.outputTopic | quote }}
Expand Down
2 changes: 0 additions & 2 deletions charts/producer-app/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,6 @@ streams:
commandLine: {}
# MY_CLI_PARAM: "foo-bar"

debug: false

env: {}
# MY_ENV_VARIABLE: foo-bar

Expand Down
6 changes: 1 addition & 5 deletions charts/streams-app-cleanup-job/templates/job.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ spec:
- name: ENV_PREFIX
value: {{ .Values.configurationEnvPrefix }}_
{{- range $key, $value := .Values.streams.config }}
- name: {{ printf "STREAMS_%s" $key | replace "." "_" | upper | quote }}
- name: {{ printf "KAFKA_%s" $key | replace "." "_" | upper | quote }}
value: {{ $value | quote }}
{{- end }}
{{- if hasKey .Values.streams "bootstrapServers" }}
Expand All @@ -76,10 +76,6 @@ spec:
- name: "{{ .Values.configurationEnvPrefix }}_SCHEMA_REGISTRY_URL"
value: {{ .Values.streams.schemaRegistryUrl | quote }}
{{- end }}
{{- if hasKey .Values "debug" }}
- name: "{{ .Values.configurationEnvPrefix }}_DEBUG"
value: {{ .Values.debug | quote }}
{{- end }}
{{- if and (hasKey .Values.streams "inputTopics") (.Values.streams.inputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_INPUT_TOPICS"
value: {{ .Values.streams.inputTopics | join "," | quote }}
Expand Down
2 changes: 0 additions & 2 deletions charts/streams-app-cleanup-job/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,6 @@ streams:
commandLine: {}
# MY_CLI_PARAM: "foo-bar"

debug: false

env: {}
# MY_ENV_VARIABLE: foo-bar
#
Expand Down
19 changes: 9 additions & 10 deletions charts/streams-app/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,16 +67,15 @@ Alternatively, a YAML file that specifies the values for the parameters can be p

### Other

| Parameter | Description | Default |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `configurationEnvPrefix` | Prefix for environment variables to use that should be parsed as command line arguments. | `APP` |
| `commandLine` | Map of command line arguments passed to the streams app. | `{}` |
| `debug` | Configure logging to debug | `false` |
| `env` | Custom environment variables | `{}` |
| `secrets` | Custom secret environment variables. Prefix with `configurationEnvPrefix` in order to pass secrets to command line or prefix with `STREAMS_` to pass secrets to Kafka Streams configuration. E.g., `APP_MY_PARAM` would be passed as `--my-param` and `STREAMS_MAX_POLL_TIMEOUT_MS` would be translated to `max.poll.timeout.ms`. | `{}` |
| `secretRefs` | Inject existing secrets as environment variables. Map key is used as environment variable name. Value consists of secret `name` and `key`. | `{}` |
| `secretFilesRefs` | Mount existing secrets as volumes | `[]` |
| `files` | Map of files to mount for the app. File will be mounted as `$value.mountPath/$key`. `$value.content` denotes file content (recommended to be used with `--set-file`). | `{}` |
| Parameter | Description | Default |
|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|
| `configurationEnvPrefix` | Prefix for environment variables to use that should be parsed as command line arguments. | `APP` |
| `commandLine` | Map of command line arguments passed to the streams app. | `{}` |
| `env` | Custom environment variables | `{}` |
| `secrets` | Custom secret environment variables. Prefix with `configurationEnvPrefix` in order to pass secrets to command line or prefix with `KAFKA_` to pass secrets to Kafka Streams configuration. E.g., `APP_MY_PARAM` would be passed as `--my-param` and `KAFKA_MAX_POLL_TIMEOUT_MS` would be translated to `max.poll.timeout.ms`. | `{}` |
| `secretRefs` | Inject existing secrets as environment variables. Map key is used as environment variable name. Value consists of secret `name` and `key`. | `{}` |
| `secretFilesRefs` | Mount existing secrets as volumes | `[]` |
| `files` | Map of files to mount for the app. File will be mounted as `$value.mountPath/$key`. `$value.content` denotes file content (recommended to be used with `--set-file`). | `{}` |

### JMX Configuration

Expand Down
10 changes: 3 additions & 7 deletions charts/streams-app/templates/deployment.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ spec:
- name: ENV_PREFIX
value: {{ .Values.configurationEnvPrefix }}_
{{- range $key, $value := .Values.streams.config }}
- name: {{ printf "STREAMS_%s" $key | replace "." "_" | upper | quote }}
- name: {{ printf "KAFKA_%s" $key | replace "." "_" | upper | quote }}
value: {{ $value | quote }}
{{- end }}
{{- range .Values.ports }}
Expand All @@ -97,12 +97,12 @@ spec:
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: STREAMS_APPLICATION_SERVER
- name: KAFKA_APPLICATION_SERVER
value: "$(POD_IP):{{ .containerPort }}"
{{- end }}
{{- end }}
{{- if .Values.streams.staticMembership }}
- name: STREAMS_GROUP_INSTANCE_ID
- name: KAFKA_GROUP_INSTANCE_ID
valueFrom:
fieldRef:
fieldPath: metadata.name
Expand All @@ -123,10 +123,6 @@ spec:
- name: "{{ .Values.configurationEnvPrefix }}_SCHEMA_REGISTRY_URL"
value: {{ .Values.streams.schemaRegistryUrl | quote }}
{{- end }}
{{- if hasKey .Values "debug" }}
- name: "{{ .Values.configurationEnvPrefix }}_DEBUG"
value: {{ .Values.debug | quote }}
{{- end }}
{{- if and (hasKey .Values.streams "inputTopics") (.Values.streams.inputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_INPUT_TOPICS"
value: {{ .Values.streams.inputTopics | join "," | quote }}
Expand Down
2 changes: 0 additions & 2 deletions charts/streams-app/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,6 @@ streams:
commandLine: {}
# MY_CLI_PARAM: "foo-bar"

debug: false

env: {}
# MY_ENV_VARIABLE: foo-bar

Expand Down
Loading

0 comments on commit 87e6bd4

Please sign in to comment.