Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resource [v1/Secret] isn't valid for cluster #288

Closed
dariusj1 opened this issue Mar 19, 2024 · 4 comments
Closed

resource [v1/Secret] isn't valid for cluster #288

dariusj1 opened this issue Mar 19, 2024 · 4 comments

Comments

@dariusj1
Copy link

I'm sorry, I must be doing something wrong, but I just can't figure out what it is.

I have a terraform project creating an EKS and setting up kubectl locally and I also want to deploy several essential resources to the cluster using manifests.

resource "kubectl_manifest" "aws_secrets_key" {
  depends_on = [module.eks]

    yaml_body = <<-EOF
      apiVersion: v1
      kind: Secret
      metadata:
        namespace: external-secrets
        name: awssm-secret
      data:
        access-key: CHANGEME
        secret-access-key: CHANGEME
    EOF
}

terraform plan generates this

  # kubectl_manifest.aws_secrets_key will be created
  + resource "kubectl_manifest" "aws_secrets_key" {
      + api_version             = "v1"
      + apply_only              = false
      + force_conflicts         = false
      + force_new               = false
      + id                      = (known after apply)
      + kind                    = "Secret"
      + live_manifest_incluster = (sensitive value)
      + live_uid                = (known after apply)
      + name                    = "awssm-secret"
      + namespace               = "external-secrets"
      + server_side_apply       = false
      + uid                     = (known after apply)
      + validate_schema         = true
      + wait_for_rollout        = true
      + yaml_body               = (sensitive value)
      + yaml_body_parsed        = <<-EOT
            apiVersion: v1
            data: (sensitive value)
            kind: Secret
            metadata:
              name: awssm-secret
              namespace: external-secrets
        EOT
      + yaml_incluster          = (sensitive value)
    }

and terraform apply fails with this

│ Error: external-secrets/awssm-secret failed to create kubernetes rest client for update of resource: resource [v1/Secret] isn't valid for cluster, check the APIVersion and Kind fields are valid
│ 
│   with kubectl_manifest.aws_secrets_key,
│   on 13_k8s_tools.tf line 20, in resource "kubectl_manifest" "aws_secrets_key":
│   20: resource "kubectl_manifest" "aws_secrets_key" {
│ 

I also tried TF_LOG=DBUG and got this

2024-03-19T12:23:12.221+0200 [DEBUG] provider.terraform-provider-aws_v5.41.0_x5: [DEBUG] Waiting for state to become: [available storage-optimization]
2024-03-19T12:23:14.488+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0: 2024/03/19 12:23:14 [DEBUG] external-secrets/awssm-secret Unstructed YAML: map[apiVersion:v1 data:map[access-key:CHANGEME secret-access-key:CHANGEME] kind:Secret metadata:map[name:awssm-secret namespace:external-secrets]]
2024-03-19T12:23:14.488+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0: 2024/03/19 12:23:14 [DEBUG] external-secrets/awssm-secret apply kubernetes resource:
2024-03-19T12:23:14.488+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0: apiVersion: v1
2024-03-19T12:23:14.488+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0: kind: Secret
2024-03-19T12:23:14.488+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0: metadata:
2024-03-19T12:23:14.488+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0:   namespace: external-secrets
2024-03-19T12:23:14.488+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0:   name: awssm-secret
2024-03-19T12:23:14.488+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0: data:
2024-03-19T12:23:14.488+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0:   access-key: CHANGEME
2024-03-19T12:23:14.488+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0:   secret-access-key: CHANGEME
2024-03-19T12:23:14.510+0200 [DEBUG] provider.terraform-provider-kubectl_v1.14.0: 2024/03/19 12:23:14 [ERROR] creating manifest failed: external-secrets/awssm-secret failed to create kubernetes rest client for update of resource: resource [v1/Secret] isn't valid for cluster, check the APIVersion and Kind fields are valid

now, if I kubectl apply -f- that same manifest, the secret is created successfully.

What am I doing wrong?

@alekc
Copy link
Contributor

alekc commented Mar 19, 2024 via email

@dariusj1
Copy link
Author

dariusj1 commented Mar 19, 2024

@alekc

using a provider which is not being updated

yeah, that's right. I used the version from README.md rather than from the registry. In registry the newest version seems to be 1.14.0

2.0.4 is in another repository which seems to be forked off the Gavin's kubectl. I'll give it a spin, thanks

kubernetes provider is much stable

perhaps, but it doesn't deal well with manifests involving CRDs. Which is the only reason why I picked the kubectl provider
hashicorp/terraform-provider-kubernetes#1367
hashicorp/terraform-provider-kubernetes#1583

@dariusj1
Copy link
Author

Upgrading the version worked. Thank you.

@alekc
Copy link
Contributor

alekc commented Mar 20, 2024

@dariusj1 thats' why I said "imho if you are dealing with standard objects". As long as it's standard (secrets, ns, deployments, etc), you will get a better time especially with diffs if using kubernetes provider and kubectl for non standard manifests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants