Add tool to update CRDs in a Kubernetes job without kubectl

[tobiasgiese]

This PR adds a tool to update CRDs based on the initial PR Mellanox/network-operator#1118.

To easily re-use this tool (as it should be build inside a container) this repository might be the best choice.

[coveralls]

Pull Request Test Coverage Report for Build 11774740600

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 37 of 105 (35.24%) changed or added relevant lines in 1 file are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-1.7%) to 62.142%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
cmd/apply-crds/main.go	37	105	35.24%

Totals
Change from base Build 11621404594:	-1.7%
Covered Lines:	1085
Relevant Lines:	1746

---

💛 - Coveralls

[adrianchiris]

LGTM !

[tobiasgiese]

Thanks for the reviews. I am not able to merge this PR as I don‘t have write access. I think only you @cdesiniotis and @adrianchiris are able to do this

[tariq1890]

Thanks for raising this PR @tobiasgiese . This is indeed very useful and a great alternative to using the kubectl binaries for creating/updating CRDs

Do we also plan on adding the Dockerfile manifest here?

[tobiasgiese]

Do we also plan on adding the Dockerfile manifest here?

We can show an example, but this Dockerfile cannot be re-used.
An example implementation is here:

• build stage https://github.com/Mellanox/network-operator/blob/master/Dockerfile#L44-L74
• copy https://github.com/Mellanox/network-operator/blob/master/Dockerfile#L81-L82

The problelm is that we need the CRD manifests inside the docker containers. We CAN mount them as configmaps, but this is probably not a good idea IMHO.
We can also add a Dockerfile and push the container to the GHCR to reuse the image, but I don't know if this is already prepared for this repo?

The current idea is to re-use it inside the repo via a custom package and build the container as mentioned above, e.g.:

package main

import (
	applycrds "github.com/nvidia/k8s-operator-libs/cmd/apply-crds"
)

func main() {
	applycrds.Run()
}

We can/should improve the readme for sure, it was just a first shot :)

[tariq1890]

Thanks for all the details @tobiasgiese! I've approved this PR for now.

The problelm is that we need the CRD manifests inside the docker containers. We CAN mount them as configmaps, but this is probably not a good idea IMHO.

What if we provided the raw link to the CRD files we push on github as a parameter to this app? As long as we provide a raw github link from a tag, it should be stable enough as an input and CRD source. If you think this is a good idea, we could implement this in a follow-up

[tobiasgiese]

What if we provided the raw link to the CRD files we push on github as a parameter to this app? As long as we provide a raw github link from a tag, it should be stable enough as an input and CRD source.

That is a good idea @tariq1890, but on the other hand it has some downsides if you think about Helm Chart dependencies.

• If you have multiple deps you have to manage those versions somehow. It might be possible to get the app version via the helm release values.
• Not every Helm Chart is releasing the CRDs in the repository.
• There is Github rate limiting that might affect the upgrade.
• Some companies do not have DIA (direct internet access) for their servers/Kubernetes clusters (or even air gapped systems). Downloading the CRDs here will not work.

Thus, I think it might be possible to add the feature, but the core-feature should be stay with local files IMHO :)

Appendix: In a downstream repo we will copy the CRDs in our Dockerfile via

RUN apk add yq

WORKDIR /workspace

COPY ./deploy/helm/operator deploy

RUN mkdir -p \
  /workspace/crds/operator \
  /workspace/crds/node-feature-discovery

# Helm show crds does not support adding only the local CRDs but will add all CRD of all dependencies, so we need to manually concatenate them.
RUN yq deploy/crds/* > /workspace/crds/operator/operator.yaml

# Add the CRDs from the helm charts separately to be able to opt-out.
RUN helm show crds deploy/charts/node-feature-discovery*.tgz > /workspace/crds/node-feature-discovery/node-feature-discovery.yaml

and as upgrade-crd-hook.yaml

containers:
  - name: upgrade-crd
    image: example.com/operator
    imagePullPolicy: IfNotPresent
    command:
      - /apply-crds
    args:
        - --crds-dir=/crds/operator
  {{ if index .Values "node-feature-discovery" "enabled" }}
        - --crds-dir=/crds/node-feature-discovery
  {{- end }}