Skip to content

Commit

Permalink
Rename to labeled
Browse files Browse the repository at this point in the history
  • Loading branch information
philipp94831 committed Jul 26, 2024
1 parent b51430c commit 7c66ffe
Show file tree
Hide file tree
Showing 22 changed files with 147 additions and 143 deletions.
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,13 +121,15 @@ The following configuration options are available:

- `--error-topic`: A topic to write errors to

- `--named-input-topics`: Additional named input topics if you need to specify multiple topics with different message
- `--labeled-input-topics`: Additional labeled input topics if you need to specify multiple topics with different
message
types (`<String=String>[,<String=String>...]`)

- `--named-input-patterns`: Additional named input patterns if you need to specify multiple topics with different
- `--labeled-input-patterns`: Additional labeled input patterns if you need to specify multiple topics with different
message types (`<String=String>[,<String=String>...]`)

- `--named-output-topics`: Additional named output topics if you need to specify multiple topics with different message
- `--labeled-output-topics`: Additional labeled output topics if you need to specify multiple topics with different
message
types (`String=String>[,<String=String>...]`)

- `--volatile-group-instance-id`: Whether the group instance id is volatile, i.e., it will change on a Streams shutdown.
Expand Down Expand Up @@ -198,7 +200,7 @@ The following configuration options are available:

- `--output-topic`: The output topic

- `--named-output-topics`: Additional named output topics (`String=String>[,<String=String>...]`)
- `--labeled-output-topics`: Additional labeled output topics (`String=String>[,<String=String>...]`)

Additionally, the following commands are available:

Expand Down
6 changes: 3 additions & 3 deletions charts/producer-app-cleanup-job/templates/job.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,9 @@ spec:
- name: "{{ .Values.configurationEnvPrefix }}_OUTPUT_TOPIC"
value: {{ .Values.kafka.outputTopic | quote }}
{{- end }}
{{- if and (hasKey .Values.kafka "namedOutputTopics") (.Values.kafka.namedOutputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_NAMED_OUTPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.namedOutputTopics }}{{ $key }}={{ $value }},{{- end }}"
{{- if and (hasKey .Values.kafka "labeledOutputTopics") (.Values.kafka.labeledOutputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_LABELED_OUTPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.labeledOutputTopics }}{{ $key }}={{ $value }},{{- end }}"
{{- end }}
{{- range $key, $value := .Values.secrets }}
- name: "{{ $key }}"
Expand Down
4 changes: 2 additions & 2 deletions charts/producer-app-cleanup-job/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ kafka:
# Note that YAML may convert large integers to scientific notation. Use Strings to avoid this.
# max.request.size: "1000000"
# outputTopic: output
namedOutputTopics: {}
# role: output
labeledOutputTopics: {}
# label: output

commandLine: {}
# MY_CLI_PARAM: "foo-bar"
Expand Down
14 changes: 7 additions & 7 deletions charts/producer-app/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,13 @@ Alternatively, a YAML file that specifies the values for the parameters can be p

### Streams

| Parameter | Description | Default |
|---------------------------|------------------------------------------------------------------------------------------------------------|---------|
| `kafka.brokers` | Comma separated list of Kafka brokers to connect to. | |
| `kafka.schemaRegistryUrl` | URL of Schema Registry to connect to. | `null` |
| `kafka.config` | Configurations for your [Kafka producer app](https://kafka.apache.org/documentation/#producerconfigs). | `{}` |
| `kafka.outputTopic` | Output topic for your producer application. | |
| `kafka.namedOutputTopics` | Map of additional named output topics if you need to specify multiple topics with different message types. | `{}` |
| Parameter | Description | Default |
|-----------------------------|--------------------------------------------------------------------------------------------------------------|---------|
| `kafka.brokers` | Comma separated list of Kafka brokers to connect to. | |
| `kafka.schemaRegistryUrl` | URL of Schema Registry to connect to. | `null` |
| `kafka.config` | Configurations for your [Kafka producer app](https://kafka.apache.org/documentation/#producerconfigs). | `{}` |
| `kafka.outputTopic` | Output topic for your producer application. | |
| `kafka.labeledOutputTopics` | Map of additional labeled output topics if you need to specify multiple topics with different message types. | `{}` |

### Other

Expand Down
6 changes: 3 additions & 3 deletions charts/producer-app/templates/pod.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ spec:
- name: "{{ .Values.configurationEnvPrefix }}_OUTPUT_TOPIC"
value: {{ .Values.kafka.outputTopic | quote }}
{{- end }}
{{- if and (hasKey .Values.kafka "namedOutputTopics") (.Values.kafka.namedOutputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_NAMED_OUTPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.namedOutputTopics }}{{ $key }}={{ $value }},{{- end }}"
{{- if and (hasKey .Values.kafka "labeledOutputTopics") (.Values.kafka.labeledOutputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_LABELED_OUTPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.labeledOutputTopics }}{{ $key }}={{ $value }},{{- end }}"
{{- end }}
{{- range $key, $value := .Values.secrets }}
- name: "{{ $key }}"
Expand Down
4 changes: 2 additions & 2 deletions charts/producer-app/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ kafka:
# Note that YAML may convert large integers to scientific notation. Use Strings to avoid this.
# max.request.size: "1000000"
# outputTopic: output
namedOutputTopics: {}
# role: output
labeledOutputTopics: {}
# label: output

commandLine: {}
# MY_CLI_PARAM: "foo-bar"
Expand Down
18 changes: 9 additions & 9 deletions charts/streams-app-cleanup-job/templates/job.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -92,18 +92,18 @@ spec:
- name: "{{ .Values.configurationEnvPrefix }}_ERROR_TOPIC"
value: {{ .Values.kafka.errorTopic | quote }}
{{- end }}
{{- if and (hasKey .Values.kafka "namedOutputTopics") (.Values.kafka.namedOutputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_NAMED_OUTPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.namedOutputTopics }}{{ $key }}={{ $value }},{{- end }}"
{{- if and (hasKey .Values.kafka "labeledOutputTopics") (.Values.kafka.labeledOutputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_LABELED_OUTPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.labeledOutputTopics }}{{ $key }}={{ $value }},{{- end }}"
{{- end }}
{{- $delimiter := ";" }}
{{- if and (hasKey .Values.kafka "namedInputTopics") (.Values.kafka.namedInputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_NAMED_INPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.namedInputTopics }}{{ $key }}={{ $value | join $delimiter }},{{- end }}"
{{- if and (hasKey .Values.kafka "labeledInputTopics") (.Values.kafka.labeledInputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_LABELED_INPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.labeledInputTopics }}{{ $key }}={{ $value | join $delimiter }},{{- end }}"
{{- end }}
{{- if and (hasKey .Values.kafka "namedInputPatterns") (.Values.kafka.namedInputPatterns) }}
- name: "{{ .Values.configurationEnvPrefix }}_NAMED_INPUT_PATTERNS"
value: "{{- range $key, $value := .Values.kafka.namedInputPatterns }}{{ $key }}={{ $value }},{{- end }}"
{{- if and (hasKey .Values.kafka "labeledInputPatterns") (.Values.kafka.labeledInputPatterns) }}
- name: "{{ .Values.configurationEnvPrefix }}_LABELED_INPUT_PATTERNS"
value: "{{- range $key, $value := .Values.kafka.labeledInputPatterns }}{{ $key }}={{ $value }},{{- end }}"
{{- end }}
{{- range $key, $value := .Values.secrets }}
- name: "{{ $key }}"
Expand Down
12 changes: 6 additions & 6 deletions charts/streams-app-cleanup-job/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,16 @@ kafka:
inputTopics: []
# - input
# - input2
namedInputTopics: {}
# role:
labeledInputTopics: {}
# label:
# - input
# - input2
# inputPattern: .*-input
namedInputPatterns: {}
# role: .*-input
labeledInputPatterns: {}
# label: .*-input
# outputTopic: output
namedOutputTopics: {}
# role: output
labeledOutputTopics: {}
# label: output
# errorTopic: error
deleteOutput: false

Expand Down
26 changes: 13 additions & 13 deletions charts/streams-app/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,19 +50,19 @@ Alternatively, a YAML file that specifies the values for the parameters can be p

### Streams

| Parameter | Description | Default |
|----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|
| `kafka.brokers` | Comma separated list of Kafka brokers to connect to. | |
| `kafka.schemaRegistryUrl` | URL of Schema Registry to connect to. | `null` |
| `kafka.staticMembership` | Whether to use [Kafka Static Group Membership](https://cwiki.apache.org/confluence/display/KAFKA/KIP-345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances). | `false` |
| `kafka.config` | Configurations for your [Kafka Streams app](https://kafka.apache.org/documentation/#streamsconfigs). | `{}` |
| `kafka.inputTopics` | List of input topics for your streams application. | `[]` |
| `kafka.namedInputTopics` | Map of additional named input topics if you need to specify multiple topics with different message types. | `{}` |
| `kafka.inputPattern` | Input pattern of topics for your streams application. | |
| `kafka.namedInputPatterns` | Map of additional named input patterns if you need to specify multiple topics with different message types. | `{}` |
| `kafka.outputTopic` | Output topic for your streams application. | |
| `kafka.namedOutputTopics` | Map of additional named output topics if you need to specify multiple topics with different message types. | `{}` |
| `kafka.errorTopic` | Error topic for your streams application. | |
| Parameter | Description | Default |
|------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|
| `kafka.brokers` | Comma separated list of Kafka brokers to connect to. | |
| `kafka.schemaRegistryUrl` | URL of Schema Registry to connect to. | `null` |
| `kafka.staticMembership` | Whether to use [Kafka Static Group Membership](https://cwiki.apache.org/confluence/display/KAFKA/KIP-345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances). | `false` |
| `kafka.config` | Configurations for your [Kafka Streams app](https://kafka.apache.org/documentation/#streamsconfigs). | `{}` |
| `kafka.inputTopics` | List of input topics for your streams application. | `[]` |
| `kafka.labeledInputTopics` | Map of additional labeled input topics if you need to specify multiple topics with different message types. | `{}` |
| `kafka.inputPattern` | Input pattern of topics for your streams application. | |
| `kafka.labeledInputPatterns` | Map of additional labeled input patterns if you need to specify multiple topics with different message types. | `{}` |
| `kafka.outputTopic` | Output topic for your streams application. | |
| `kafka.labeledOutputTopics` | Map of additional labeled output topics if you need to specify multiple topics with different message types. | `{}` |
| `kafka.errorTopic` | Error topic for your streams application. | |

### Other

Expand Down
18 changes: 9 additions & 9 deletions charts/streams-app/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -139,18 +139,18 @@ spec:
- name: "{{ .Values.configurationEnvPrefix }}_ERROR_TOPIC"
value: {{ .Values.kafka.errorTopic | quote }}
{{- end }}
{{- if and (hasKey .Values.kafka "namedOutputTopics") (.Values.kafka.namedOutputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_NAMED_OUTPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.namedOutputTopics }}{{ $key }}={{ $value }},{{- end }}"
{{- if and (hasKey .Values.kafka "labeledOutputTopics") (.Values.kafka.labeledOutputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_LABELED_OUTPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.labeledOutputTopics }}{{ $key }}={{ $value }},{{- end }}"
{{- end }}
{{- $delimiter := ";" }}
{{- if and (hasKey .Values.kafka "namedInputTopics") (.Values.kafka.namedInputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_NAMED_INPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.namedInputTopics }}{{ $key }}={{ $value | join $delimiter }},{{- end }}"
{{- if and (hasKey .Values.kafka "labeledInputTopics") (.Values.kafka.labeledInputTopics) }}
- name: "{{ .Values.configurationEnvPrefix }}_LABELED_INPUT_TOPICS"
value: "{{- range $key, $value := .Values.kafka.labeledInputTopics }}{{ $key }}={{ $value | join $delimiter }},{{- end }}"
{{- end }}
{{- if and (hasKey .Values.kafka "namedInputPatterns") (.Values.kafka.namedInputPatterns) }}
- name: "{{ .Values.configurationEnvPrefix }}_NAMED_INPUT_PATTERNS"
value: "{{- range $key, $value := .Values.kafka.namedInputPatterns }}{{ $key }}={{ $value }},{{- end }}"
{{- if and (hasKey .Values.kafka "labeledInputPatterns") (.Values.kafka.labeledInputPatterns) }}
- name: "{{ .Values.configurationEnvPrefix }}_LABELED_INPUT_PATTERNS"
value: "{{- range $key, $value := .Values.kafka.labeledInputPatterns }}{{ $key }}={{ $value }},{{- end }}"
{{- end }}
{{- range $key, $value := .Values.secrets }}
- name: "{{ $key }}"
Expand Down
6 changes: 3 additions & 3 deletions charts/streams-app/templates/scaled-object.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ spec:
idleReplicaCount: {{ .Values.autoscaling.idleReplicas }}
{{- end }}
triggers:
{{- if not (or .Values.kafka.inputTopics .Values.autoscaling.internalTopics .Values.autoscaling.topics .Values.kafka.namedInputTopics .Values.autoscaling.additionalTriggers) }}
{{- fail "To use autoscaling, you must define one of .Values.kafka.inputTopics, .Values.autoscaling.internalTopics, .Values.autoscaling.topics, .Values.kafka.namedInputTopics or .Values.autoscaling.additionalTriggers" }}
{{- if not (or .Values.kafka.inputTopics .Values.autoscaling.internalTopics .Values.autoscaling.topics .Values.kafka.labeledInputTopics .Values.autoscaling.additionalTriggers) }}
{{- fail "To use autoscaling, you must define one of .Values.kafka.inputTopics, .Values.autoscaling.internalTopics, .Values.autoscaling.topics, .Values.kafka.labeledInputTopics or .Values.autoscaling.additionalTriggers" }}
{{- end}}
# todo: concat .Values.kafka.inputTopics and .Values.autoscaling.topics to
# minimize number of loops when we don't need to support helm 2 anymore
Expand Down Expand Up @@ -58,7 +58,7 @@ spec:
lagThreshold: {{ $root.Values.autoscaling.lagThreshold | quote }}
offsetResetPolicy: {{ $root.Values.autoscaling.offsetResetPolicy }}
{{- end }}
{{- range $key, $value := .Values.kafka.namedInputTopics }}
{{- range $key, $value := .Values.kafka.labeledInputTopics }}
{{- range $topic := $value }}
- type: kafka
metadata:
Expand Down
14 changes: 7 additions & 7 deletions charts/streams-app/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,16 +34,16 @@ kafka:
inputTopics: []
# - input
# - input2
namedInputTopics: {}
# role:
labeledInputTopics: {}
# label:
# - input
# - input2
# inputPattern: .*-input
namedInputPatterns: {}
# role: .*-input
labeledInputPatterns: {}
# label: .*-input
# outputTopic: output
namedOutputTopics: {}
# role: output
labeledOutputTopics: {}
# label: output
# errorTopic: error

commandLine: {}
Expand Down Expand Up @@ -105,7 +105,7 @@ autoscaling:
minReplicas: 0
maxReplicas: 1
# idleReplicas: 0
## all topics from kafka.inputTopics and kafka.namedInputTopics are automatically taken
## all topics from kafka.inputTopics and kafka.labeledInputTopics are automatically taken
## only use the 'internalTopics' option for adding internal topics, i.e., auto-generated topics by Kafka Streams. Consumer group name will automatically be added as a prefix
internalTopics: []
# - bar-repartition # results in foo-bar-repartition
Expand Down
Loading

0 comments on commit 7c66ffe

Please sign in to comment.