Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubernetes-ingress: Make controller.config templatable #278

Open
phihos opened this issue Dec 23, 2024 · 2 comments · May be fixed by #279
Open

kubernetes-ingress: Make controller.config templatable #278

phihos opened this issue Dec 23, 2024 · 2 comments · May be fixed by #279
Assignees
Labels
enhancement New feature or request

Comments

@phihos
Copy link
Contributor

phihos commented Dec 23, 2024

Hi,

I thought about what @dkorunic said in this comment. Especially

We don't really want pod restarts either, whatever helm mentions as best practice, it's not necessarily applicable to everything, especially for Ingress Controller services that we want to be restarted as little as possible.

The use-case in #275 was to make global, frontend or backend config snippets parametrizable like this:

# values-common.yaml
kubernetes-ingress:
  controller:
    extraEnvFrom:
      - configMapRef:
          name: my-env-vars
    podAnnotations: |
      env-var-checksum: {{ $.Values.customEnvVars | toYaml | sha256sum }}
    config:
      frontend-config-snippet: |
        # ...
        .if streq("${.SECTION}",https)
          maxconn "$MAXCONN_FRONTEND_HTTPS"
        .endif
        # ...
  customEnvVars:
    # default value to be overridden
    MAXCONN_FRONTEND_HTTPS: 30000

As @dkorunic pointed out this will cause pods to restart which we want to avoid. Instead we could make controller.config templatable and achieve the same without restart:

# values-common.yaml
kubernetes-ingress:
  controller:
    config: |
      frontend-config-snippet: |
        # ...
        .if streq("${.SECTION}",https)
          maxconn {{ .Values.customVars.maxconnFrontendHttps | required "customVars.maxconnFrontendHttps must be defined" }}
        .endif
        # ...
  customVars:
    # default value to be overridden
    maxconnFrontendHttps: 30000

The config is equivalent to the snippet above, but we now have additional validation preventing empty values to be inserted.
Whenever the config changes, the configmap is re-ingested by the IC and haproxy is reloaded. No restart necessary.

The same use-cases as in #275 apply here: Parametrizing config snippets so single variables can be different on different deployments, better separation of config and sets of tunables that are of interest to different teams in an organization (see this example).

This would not make #275 obsolete as templatable podAnnotations are still needed when changing haproxy-auxiliary-configmap (docs) as is can be only refreshed via pod restart.

@dkorunic dkorunic self-assigned this Dec 24, 2024
@dkorunic dkorunic added the enhancement New feature or request label Dec 24, 2024
@phihos
Copy link
Contributor Author

phihos commented Dec 30, 2024

@dkorunic I can also implement that with a test if that feature is fine with you.

@dkorunic
Copy link
Member

@phihos That would be great, if you are willing to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants