Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Helm template storage backend logic #685

Merged
merged 1 commit into from
Mar 24, 2022

Conversation

mszostok
Copy link
Member

@mszostok mszostok commented Mar 22, 2022

Description

Changes proposed in this pull request:

  • Add Helm template storage backend logic

Testing

  1. Create cluster:
    export DISABLE_MONITORING_INSTALLATION=true
    export ENABLE_POPULATOR=true
    export CLUSTER_TYPE=k3d
    export DISABLE_K3D_REGISTRY=false
    make dev-cluster
    
  2. Install Helm backend via UI
  3. Upgrade Helm backend release (new chart is not yet released):
    helm upgrade helm-storage-backend-1648070484 -n default ./deploy/kubernetes/charts/helm-storage-backend --set=global.containerRegistry.path="ghcr.io/capactio/pr" --set=global.containerRegistry.overrideTag="PR-685" --wait --reuse-values
    Update helm release name.
  4. Until Handle multiple backends for the TypeInstance #657 is not merge, run:
    kubectl set image deploy/capact-hub-local hub-local=ghcr.io/capactio/pr/hub-js:PR-657
    
  5. Install sample Helm release via Runner:
    RUNNER_CONTEXT_PATH=cmd/helm-runner/example-input/context.yaml \
     RUNNER_ARGS_PATH=cmd/helm-runner/example-input/install-args.yaml \
     RUNNER_LOGGER_DEV_MODE=true \
     RUNNER_COMMAND="install" \
     go run cmd/helm-runner/main.go
  6. Get ID of cap.type.helm.template.storage:0.1.0
  7. Run test:
    BACKEND_ID="ecdfece6-da75-483f-a3ca-24246ed3308d" go test ./pkg/hub/client/ -run TestThatShowcaseExternalStorage -v -count 1

Unfortunately, in my case, it's quite slow. Probably we will need to debug it further. Currently, for my setup to work properly, it requires 1CPU.

The general problem is that the PostgreSQL has as a dependency “common” chart (small size). We need to download it (in general deps charts) as it won't be available by default. The LocateChart method from Helm project, that is able to download this chart, starts with downloading an index file which in case of Bitnami has >3MB. This needs to be decoded and as a result it requires a much higher CPU limit.

You could say, that this is a special case, but unfortunately the PostgreSQL is quite often used in our examples. It needs to work properly.

Additionally, Helm methods don't support context, so we are not able to cancel any ongoing operation even if user cancelled the request and no one will wait for the response.

Related issue(s)

Fix #672

@mszostok mszostok added enhancement New feature or request area/engine Relates to Engine area/hub Relates to Hub labels Mar 22, 2022
@mszostok mszostok force-pushed the helm-storage/templates branch 6 times, most recently from acb8f0f to 4ab4266 Compare March 23, 2022 22:55
@pkosiec pkosiec self-assigned this Mar 24, 2022
Copy link
Member

@pkosiec pkosiec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works well, I've just tested that E2E on my PR (capactio/hub-manifests#64). Just a few minor comments 🙂

internal/helm-storage-backend/template.go Outdated Show resolved Hide resolved
@mszostok mszostok force-pushed the helm-storage/templates branch 2 times, most recently from d789d38 to bd473a2 Compare March 24, 2022 22:24
@mszostok mszostok enabled auto-merge (squash) March 24, 2022 22:28
@mszostok mszostok merged commit bf8d006 into capactio:main Mar 24, 2022
@mszostok mszostok deleted the helm-storage/templates branch March 28, 2022 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/engine Relates to Engine area/hub Relates to Hub enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement Helm template handler for Helm storage backend
2 participants