Incompatibility Issue with controller-runtime Version Pinning and envtest Setup - It is requiring go 1.22 when the latest release still using go 1.21 as old project versions #2744

camilamacedo86 · 2024-03-29T09:24:32Z

Problem Description

We've encountered an issue where our environment cannot pin a specific version of controller-runtime, leading to compatibility issues with earlier versions of kubebuilder when setting up envtest. This incompatibility prevents our projects from maintaining a consistent development environment, particularly affecting our CI/CD pipelines.

Error Output

While attempting to set up envtest (go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest), we're met with the following error message due to the version incompatibility besides the LATEST v0.17.2 does not support go 1.22 and old projects not be using it yet as well.

go: downloading sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20240327193027-21368602d84b
go: downloading sigs.k8s.io/controller-runtime v0.17.2
go: sigs.k8s.io/controller-runtime/tools/setup-envtest@latest: sigs.k8s.io/controller-runtime/tools/[email protected] requires go >= 1.22.0 (running go 1.21.8; GOTOOLCHAIN=local)

This error highlights the requirement for Go version >= 1.22.0, while we are currently using Go 1.21.8.

Ideal Solution (controller-runtime provide a envtest setup solution that allows us pin the tag releases)

Ideally, we need a way to setup envtest that allows version pinning per tag release. This would enable our community to manage dependencies more effectively and ensure compatibility across different versions of kubebuilder and controller-runtime. Note that in many scenarios we might need to do this setup in shell scripts.

Alternative Solution (how we could fix it and which seems the best available approach so far)

Use the release branch, example : go install sigs.k8s.io/controller-runtime/tools/setup-envtest@release-17

The text was updated successfully, but these errors were encountered:

camilamacedo86 · 2024-03-29T09:38:44Z

@joelanford @varshaprasad96 @vincepri @sbueringer

I'm reaching out to gather insights or possible solutions, as we're currently facing issues with the existing setup. Our immediate goal is to enhance the experience for Kubebuilder users and resolve CI failures by utilizing the release branch of controller-runtime.

Here's the snippet from our current approach:

ENVTEST ?= $(LOCALBIN)/setup-envtest-$(ENVTEST_VERSION)
ENVTEST_VERSION ?= release-0.17
...

.PHONY: envtest
envtest: $(ENVTEST) ## Download setup-envtest locally if necessary.
$(ENVTEST): $(LOCALBIN)
	$(call go-install-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest,$(ENVTEST_VERSION))

While this method serves as a temporary fix, a more ideal solution would involve pinning specific tag versions to ensure stability and predictability. The reliance on the release branch introduces a risk, especially if a release branch is not created as part of the controller-runtime's release process. This potential inconsistency could adversely affect Kubebuilder and the broader community.

lukas016 · 2024-04-01T17:29:11Z

proper versioning of setup-env is solving here #2646
but maintainers don't react.

…ntime#2744 Signed-off-by: Michael Valdron <[email protected]>

…ntime#2744 (#87) Signed-off-by: Michael Valdron <[email protected]>

* feat: support schemas * fix: alignements * fix: merge * fix: merge * fix: setupenvtest version kubernetes-sigs/controller-runtime#2744 * fix: lint

sbueringer · 2024-04-19T15:03:00Z

Hey folks,
we are currently experimenting a bit on setup-envtest because the envtest binaries are today hosted on the Google-owned kubebuilder repository.

Going forward we will start hosting the envtest binaries on controller-tools releases. We currently also plan to move setup-envtest to controller-tools as it fits better there. As part of that we will build & publish the setup-envtest binary on controller-tools releases in the future. This should also give proper versioning.
#2646 (comment)

Let me know if this covers the issue.

k8s-triage-robot · 2024-07-18T15:25:41Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

camilamacedo86 · 2024-07-18T18:04:22Z

Hi @sbueringer,

Issue comment

Thank you for your input on this issue. However, the challenge we're facing isn't fully addressed by the current solution.

The problem

The core issue relates to the versioning of setup-envtest itself. Specifically, we are unable to run:

go install sigs.k8s.io/controller-runtime/tools/setup-envtest@<controller-runtime-tag-release>

This means we can't align the setup-envtest version precisely with the controller-runtime version updates. We faced some incompatible scenarios with the latest.

Solution (Best approach find so far)

Then, to circumvent potential problems, we are currently scaffolding kubebuilder with:

go install sigs.k8s.io/controller-runtime/tools/setup-envtest@<branch-release>
// We can only use a branch release OR latest.

The downside of using the branch is if a release is made from master without updating/creating the branch, it could lead to issues, although so far, this method has helped us avoid any major problems.

Conclusion

However, I think we can recommend the branch release as a workaround for these limitations as a satisfactory solution. In this way, since we've switched to using the branch release, and we haven't encountered significant issues and I'm unsure how if we have a technical solution to could effectively tag and release envtest-setup to match controller-runtime. Therefore, I am comfortable with closing the issue on those terms.

So I will close it based on the above, if anyone does not agree, they can open a new issue pointing to this one and propose a better solution.

Thank you for considering this perspective.

sbueringer · 2024-07-19T06:36:31Z

As part of that we will build & publish the setup-envtest binary on controller-tools releases in the future. This should also give proper versioning.

This plan slightly changed (at least in my head). I would now publish setup-envtest binaries on controller-runtime releases. So my question was something like: If every controller-runtime release has attachments with setup-envtest binaries for various OS / ARCH, would this address your problem? (the binary would have to be downloaded instead of compiled locally, but that seems more of a benefit to me)

sbueringer · 2024-08-09T06:17:49Z

Please see #2911
With this PR, setup-envtest binaries for every upcoming controller-runtime release can simply be downloaded from release attachments

sbueringer · 2024-08-15T07:22:15Z

Now starting with the v0.19 release setup-envtest binaries are available via release attachments (see e.g. https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.19.0)

I would recommend using these binaries if:

a specific setup-envtest version is required
folks have trouble compiling the binary themselves

…ntime#2744 (devfile#87) Signed-off-by: Michael Valdron <[email protected]> Signed-off-by: thepetk <[email protected]>

…c setup - Installs ENVTEST binaries in the project’s `./bin/envtest` directory for isolation. - Configures `setup-envtest` binary to be downloaded based on the controller-runtime release version to ensure compatibility with the project’s dependencies. - Updates `suite_test.go` to dynamically locate and set the binary assets directory for seamless IDE debugging. - Introduces `getFirstFoundEnvTestBinaryDir` to detect ENVTEST binaries in the project directory when running tests outside the Makefile context. 1. **Ease of IDE Debugging**: - By setting the `BinaryAssetsDirectory` dynamically, contributors can debug tests in their preferred IDEs (e.g., GoLand, VSCode) without requiring additional environment configuration such as setting `KUBEBUILDER_ASSETS`. 2. **Project-Specific Binaries**: - Installing binaries inside the project avoids conflicts with other projects or system-wide installations, particularly useful for contributors with restricted environments (e.g., corporate laptops). 3. **Consistency**: - Using `setup-envtest` from the matching controller-runtime release avoids compatibility issues, such as those outlined in [#2744](kubernetes-sigs/controller-runtime#2744) and [#2646](kubernetes-sigs/controller-runtime#2646). 4. **Simplified Contributor Onboarding**: - New contributors often face a learning curve to: - Understand ENVTEST and its dependency on binaries. - Configure `KUBEBUILDER_ASSETS` manually. - Set up their IDE for debugging tests. - This setup minimizes friction and ensures tests can be run consistently across different environments. While setting `KUBEBUILDER_ASSETS` is sufficient for experienced contributors, this change makes it easier for newcomers and supports debugging directly in IDEs without additional configuration.

camilamacedo86 mentioned this issue Mar 29, 2024

🐛 Fix compatibility issues encountered while setting up envtest by using the controller-runtime branch releases (unblock CI) kubernetes-sigs/kubebuilder#3824

Merged

mjlshen mentioned this issue Mar 30, 2024

Update push steps to use containerized skopeo openshift/boilerplate#343

Merged

ushitora-anqou mentioned this issue Apr 1, 2024

pin setup-envtest to go 1.20 topolvm/pie#115

Merged

ushitora-anqou mentioned this issue Apr 4, 2024

pin setup-envtest to go 1.20 topolvm/pvc-autoresizer#250

Merged

pmalek mentioned this issue Apr 5, 2024

Cannot run / install specific versions of setup-envtest due to missing submodule tag #2759

Closed

michael-valdron mentioned this issue Apr 11, 2024

Registry operator fails on unit tests devfile/api#1519

Closed

michael-valdron added a commit to michael-valdron/devfile-registry-operator that referenced this issue Apr 11, 2024

fix setup-envtest version to workaround kubernetes-sigs/controller-ru…

6fdf405

…ntime#2744 Signed-off-by: Michael Valdron <[email protected]>

michael-valdron mentioned this issue Apr 11, 2024

Fix setup-envtest version devfile/registry-operator#87

Merged

3 tasks

michael-valdron added a commit to devfile/registry-operator that referenced this issue Apr 11, 2024

fix setup-envtest version to workaround kubernetes-sigs/controller-ru…

c385aba

…ntime#2744 (#87) Signed-off-by: Michael Valdron <[email protected]>

kobzonega mentioned this issue Apr 12, 2024

Pin envtest version to fix unit-test ydb-platform/ydb-kubernetes-operator#194

Merged

7 tasks

raffis added a commit to DoodleScheduling/db-controller that referenced this issue Apr 17, 2024

fix: setupenvtest version kubernetes-sigs/controller-runtime#2744

06ddcd4

adambkaplan mentioned this issue Jun 7, 2024

📖 Add setup-envtest Instructions for Go < 1.22 #2849

Merged

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 18, 2024

camilamacedo86 closed this as completed Jul 18, 2024

sbueringer mentioned this issue Aug 15, 2024

proper versioning for setup-envtest #2646

Closed

camilamacedo86 mentioned this issue Nov 17, 2024

🌱 Configure ENVTEST Binaries for IDE Debugging operator-framework/operator-controller#1454

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Incompatibility Issue with controller-runtime Version Pinning and envtest Setup - It is requiring go 1.22 when the latest release still using go 1.21 as old project versions #2744

Incompatibility Issue with controller-runtime Version Pinning and envtest Setup - It is requiring go 1.22 when the latest release still using go 1.21 as old project versions #2744

camilamacedo86 commented Mar 29, 2024 •

edited

Loading

camilamacedo86 commented Mar 29, 2024 •

edited

Loading

lukas016 commented Apr 1, 2024

sbueringer commented Apr 19, 2024

k8s-triage-robot commented Jul 18, 2024

camilamacedo86 commented Jul 18, 2024 •

edited

Loading

sbueringer commented Jul 19, 2024 •

edited

Loading

sbueringer commented Aug 9, 2024

sbueringer commented Aug 15, 2024 •

edited

Loading

Incompatibility Issue with controller-runtime Version Pinning and envtest Setup - It is requiring go 1.22 when the latest release still using go 1.21 as old project versions #2744

Incompatibility Issue with controller-runtime Version Pinning and envtest Setup - It is requiring go 1.22 when the latest release still using go 1.21 as old project versions #2744

Comments

camilamacedo86 commented Mar 29, 2024 • edited Loading

Problem Description

Error Output

Ideal Solution (controller-runtime provide a envtest setup solution that allows us pin the tag releases)

Alternative Solution (how we could fix it and which seems the best available approach so far)

camilamacedo86 commented Mar 29, 2024 • edited Loading

lukas016 commented Apr 1, 2024

sbueringer commented Apr 19, 2024

k8s-triage-robot commented Jul 18, 2024

camilamacedo86 commented Jul 18, 2024 • edited Loading

The problem

Solution (Best approach find so far)

Conclusion

sbueringer commented Jul 19, 2024 • edited Loading

sbueringer commented Aug 9, 2024

sbueringer commented Aug 15, 2024 • edited Loading

camilamacedo86 commented Mar 29, 2024 •

edited

Loading

camilamacedo86 commented Mar 29, 2024 •

edited

Loading

camilamacedo86 commented Jul 18, 2024 •

edited

Loading

sbueringer commented Jul 19, 2024 •

edited

Loading

sbueringer commented Aug 15, 2024 •

edited

Loading