common automatic update #36
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
mbaldessari
commented
Apr 29, 2024
mbaldessari
commented
- Set the user's passwd entry inside the container
- Upgrade helm to v3.13.2
- Drop old patch around null subkeys
- Do check for remote existance all the time
- Run validate-prereq only when not in a container
- Bump dorny/paths-filter from 2 to 3
- Add support for parsing secrets into intermediate structure and creating k8s secret objects
- Upgrade ESO to v0.9.12
- Update vault image to 1.15.5-ubi
- Use gitops-1.11 in acm as well
- Small gitops channel cleanups
- Upgrade namespaced argocd version to v1beta1
- Stop using OpenShiftControllerManager lookups
- Bump azure/setup-helm from 3 to 4
- Upgrade ESO to v0.9.13
- Upgrade vault to 1.15.6
- Bind mount /etc/pki in the wrapper
- Properly error out in preview-all when we cannot connect to the cluster
- Only include values files if they do exist in preview.sh
- Do not error out in preview when kustomize: true
- Also pass EXTRA_PLAYBOOK_OPTS environment setting
- Use oc kustomize
- Print out application being parsed for easier debugging
- Add .global.secretStore.backend in the clustergroup schema
- Fix check for secretStore.backend
- Introduce a generic parameter override via the EXTRA_ARGS env variable
- Allow overriding the oc lookups in preview.sh
- Always include values-global.yaml and values-.yaml
- Add help and message clarifying that preview has certain limits
- Add clustergroup support to preview target
- Fix preview when the application's index name is not the same as the name attribute
- Force strings in extraParametersNested
- In ACM policies do not use $ARGOCD_APP_SOURCE_ variables*
- Add support for custom CAs
- Update tests
- Support for issue #459
- Drop unused piece of schema json
- Small cleanup to remove unneeded log messages
- Support for cluster-wide proxy
- Update ESO to 0.9.14
- Update CRD from operator v0.0.44
- Expose main.experimentalCapabilities in operator-install
- Release clustergroup v0.8.3
- Support remote repoURL when previewing templates
- Moved CLUSTERGROUP declaration to restore make preview-% functionality
- Namespace argocd.argoproj.io/managed-by label issue
- Fixed indenting and duplicate entries in application-policies.yaml
- Add main.experimentalCapabilities to values.schema.json
- Release clustergroup v0.8.4
- Move the CLUSTERGROUP env variable for previews inside its section
- Fix up tests after upstream PR merge
- Drop old comments
- bug: Fix to generate OperatorGroup definition when namespaces definition has labels
- Try fallbacks for /etc/pki when it does not exist
- Fix for multiple OperatorGroup rendering
- Updated tests for CI
- Update ESO to 0.9.16
- Updated vault-helm to v0.28.0
- Use --recurse-submodules when cloning
- Drop support for IIB on OCP 4.12
- Stop saving stderr in a file when running skopeo
- Rearchitect the code so we can print out the actual error
- Check for prereqs for IIB
- Update tests after common rebase
The reason for this is somewhat multi-faceted, but boils down to the fact that openssh does not consult the $HOME variable to find .ssh/* files but only relies to the home folder entry in /etc/passwd. So what might happen is the following scenario: 1. The remote is ssh based: `origin [email protected]:validatedpatterns/industrial-edge` 2. The main Makefile invokes `git remote show origin` which triggers an ssh connection 3. The ssh connection fails because ssh ignores the $HOME variable and instead relies on the home in `getent passwd`. Which is set to: ``` fedora:*:1000:1000:fedora Cloud User:/home/fedora/industrial-edge:/bin/sh ``` 4. Newer podmans set the user's home folder automagically to the folder that is passed as current working directory (in our case we pass `-w $(pwd)`) Under these circumstances ssh connection will fail because git+ssh will look for ssh files in the current folder (aka entry in /etc/passwd): debug1: identity file /home/fedora/industrial-edge/.ssh/id_rsa type -1 debug1: identity file /home/fedora/industrial-edge/.ssh/id_rsa-cert type -1 Fix this by making sure we force an /etc/passwd entry for the user running podman that points to the $HOME directory (aka /pattern-home inside the container).
Set the user's passwd entry inside the container
This is the version we use in gitops-1.11 which is the new default
Upgrade helm to v3.13.2
Now that we switched to gitops-1.11, the helm version is recent enough that we're not affected by the subkey null bug any longer.
Drop old patch around null subkeys
At the time we disabled the `validate-origin` target when running from
inside the container as it apparently caused issues for some folks.
I think now that we run as the user inside the container, the chances of
this not working are reduced, so let's reenable this.
Tested as follows:
❯ ./pattern.sh make TARGET_ORIGIN=upstream validate-origin
Checking repository:
https://github.com/hybrid-cloud-patterns/multicloud-gitops - branch 'nonexisting': NOT FOUND
make: *** [Makefile:12: validate-origin] Error 2
❯ ./pattern.sh make TARGET_ORIGIN=upstream validate-origin
Checking repository:
https://github.com/hybrid-cloud-patterns/multicloud-gitops - branch 'main': OK
❯ ./pattern.sh make validate-origin
Checking repository:
https://github.com/mbaldessari/multicloud-gitops.git - branch 'main': OK
❯ ./pattern.sh make validate-origin
Checking repository:
https://github.com/mbaldessari/multicloud-gitops.git - branch 'nonexisting': NOT FOUND
make: *** [Makefile:12: validate-origin] Error 2
Validate origin inside podman as well
There is no point in testing the requirements when we use the container,
as we guarantee that those exist in there.
Tested as follows:
❯ make validate-prereq
make -f common/Makefile validate-prereq
make[1]: Entering directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops'
Checking prerequisites:
Check for 'git helm oc ansible': OK
Check for python-kubernetes: OK
Check for kubernetes.core collection: OK
make[1]: Leaving directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops'
❯ ./pattern.sh make validate-prereq
make -f common/Makefile validate-prereq
make[1]: Entering directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops'
Skipping prerequisites check as we're running inside a container
make[1]: Leaving directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops'
Run validate-prereq only when not in a container
Bumps [dorny/paths-filter](https://github.com/dorny/paths-filter) from 2 to 3. - [Release notes](https://github.com/dorny/paths-filter/releases) - [Changelog](https://github.com/dorny/paths-filter/blob/master/CHANGELOG.md) - [Commits](dorny/paths-filter@v2...v3) --- updated-dependencies: - dependency-name: dorny/paths-filter dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]>
…ons/dorny/paths-filter-3 Bump dorny/paths-filter from 2 to 3
k8s secret objects Ensure only push_secrets runs from vault_utils Update makefile - remove extra targets and make fix none Revert version bump as we only add fields Conditionalize check change output Start module to load parsed secrets into vault New machinery for vault secrets loading Make the linters pass again Inject policies Add some more code to test readiness to load Correct typo Add vault_hub Add vaultMount Rename new modules to v2 Update inject_field method Correct field typo Temporarily print command Add more logic Hopefully fix secret loading issue with counter Count per secret Pick stuff out of secret that we need Fix lint issue Refactor tests to use fixture constants Correctly spell exclusion for ansible-lint Provide a target to exercise legacy code path Add error exists for missing args and update docs Reverse test for override Also process base64 for generated secrets Be more explicit about what we load Test framework for loading parsed_secret data Fix linting errors Finish test suite Last linter stuff Change schema; code and tests to follow Add target_namespaces phase 1 more passing, but some still fail Passing again All pass Check the correct variable in golang-external-secrets chart Update YAML parsing to do decodes right Add tests and tighten up code for retrieving block yaml quotes Add test for kubernetes secret object and block yaml
Add support for kubernetes backend for ESO
Upgrade ESO to v0.9.12
This fixes a few CVEs. Tested on MCG.
Update vault image to 1.15.5-ubi
This is mainly for consistency reasons as the value is taken from main.gitops anyways.
Use gitops-1.11 in acm as well
Mainly for consistency reasons. gitops-1.11 is already the default
Small gitops channel cleanups
Upgrade namespaced argocd version to v1beta1
When applying the policy to install the cluster-wide argo on regional clusters, we do some lookups() on the regional cluster in order to pass the version values, domain names, etc. To get the cluster version we were using the OpenShiftControllerManager which is problematic because it does not exist on hyper-shift clusters. Let's switch to use the ClusterVersion.status.history[0].version entry. The only smaller caveat is that due to limitations in go templates + sprig functions, we cannot really take the last version only when the state is "Completed", but we simply take the last version. This means that during a cluster upgrade on a regional cluster, we will include values of the version the cluster is upgrading to, which is less than ideal, but it should eventually converge in any case. For reference the function that guarantees that the ordering of the history status in ClusterVersion is preserved is here: https://pkg.go.dev/github.com/openshift/api/config/v1#ClusterVersionStatus Tested on Lester's cluster and on a local cluster of mine. Co-Authored-By: Lester Claudio <[email protected]>
Stop using OpenShiftControllerManager lookups
Bumps [azure/setup-helm](https://github.com/azure/setup-helm) from 3 to 4. - [Release notes](https://github.com/azure/setup-helm/releases) - [Changelog](https://github.com/Azure/setup-helm/blob/main/CHANGELOG.md) - [Commits](Azure/setup-helm@v3...v4) --- updated-dependencies: - dependency-name: azure/setup-helm dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]>
…ons/azure/setup-helm-4 Bump azure/setup-helm from 3 to 4
Upgrade ESO to v0.9.13
ClusterGroup v0.8.4
Moved CLUSTERGROUP declaration to restore make preview-% functionality
This way the code is a bit less confusing and it's more obvious when/where CLUSTERGROUP is used. make preview-% still keeps working as usual and you can override things via `make CLUSTERGROUP=group-one preview-hello-world` Gotta love Makefile's idiosyncrasies around per-target variables
Small makefile tweak
bug: Fixes indenting and duplicate entries in application-policies.yaml
…ion has labels
- Added condition to check if operatorGroup key exists {{- if or $v.operatorGroup (not (hasKey $v "operatorGroup")) }}
- Default behavior is that we generate an OperatorGroup for a Namespace definition.
Small cleanups
bug: Fix to generate OperatorGroup definition when namespaces definition has labels
Try fallbacks for /etc/pki when it does not exist
- Fixed issue with rendering multiple OperatorGroup. Added the beginning (---) to signal document start. - Updated the applications.yaml to include beginning (---) to signal document start.
helper operatorgroup fix
Update ESO to 0.9.16
Updated vault-helm to v0.28.0
In case a git repo has some things split over submodules, let's clone those too. The version of git in the imperative image is currently: sh-5.1# git version git version 2.39.3 Tested with: sh-5.1# git clone --recurse-submodules --single-branch --branch main --depth 1 -- "https://github.com/validatedpatterns/multicloud-gitops" /tmp/ Cloning into '/tmp'... remote: Enumerating objects: 426, done. remote: Counting objects: 100% (426/426), done. remote: Compressing objects: 100% (343/343), done. remote: Total 426 (delta 87), reused 221 (delta 40), pack-reused 0 Receiving objects: 100% (426/426), 545.98 KiB | 1.78 MiB/s, done. Resolving deltas: 100% (87/87), done. Co-Authored-By: Sergio Garcia Martinez <[email protected]>
Use --recurse-submodules when cloning
The last job we ran with that for IIB was more than 4 months ago.
Check for prereqs for IIB
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.