Drift in `cloudflare_ruleset` for `http_request_firewall_custom` phase for computed args `id`, `last_updated`, `ref`, `version` #2690

nitrocode · 2023-08-16T22:45:56Z

Confirmation

My issue isn't already found on the issue tracker.
I have replicated my issue using the latest version of the provider and it is still present.

Terraform and Cloudflare provider version

Terraform 1.5.5
Cloudflare provider 4.12.0 latest

Affected resource(s)

cloudflare_ruleset

Terraform configuration files

resource "cloudflare_ruleset" "custom" {
  kind    = "zone"
  name    = "default"
  phase   = "http_request_firewall_custom"
  zone_id = local.zone_id

  # ... 30 rules ...

  # e.g. pre-existing rule
  rules {
    enabled     = true
    action      = "log"
    description = "<snip>"
    expression  = <<-EOT
              <snip>
            EOT
  }

  # added additional rule which caused a drift. could be any rule
  rules {
    enabled     = true
    action      = "log"
    description = "<snip>"
    expression  = <<-EOT
              any(http.request.headers["custom-header"][*] matches ".?")
            EOT
  }

Link to debug output

none

Panic output

N/A

Expected output

No perma drift

Actual output

Perma drift

Steps to reproduce

Add a new rule or modify an existing one
terraform plan

Additional factoids

This happened after we bumped the provider from 3.x to 4.x

We deleted the cloudflare_ruleset and reimported.

If we do not make a change, we see No changes as expected. Any change we make, and we see a bunch of drift between rules, all affecting arguments that we do not set such as id and version

We have 30 something rules and we see these computed arguments changing for every rule.

in 4.12.0 cloudflare provider

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # cloudflare_ruleset.custom will be updated in-place
  ~ resource "cloudflare_ruleset" "custom" {
        id      = "<snip>"
        name    = "default"
        # (3 unchanged attributes hidden)

      ~ rules {
          ~ id           = "<snip>" -> (known after apply)
          ~ last_updated = "2023-08-18 16:36:15.176813 +0000 UTC" -> (known after apply)
          ~ ref          = "<snip>" -> (known after apply)
          ~ version      = "3" -> (known after apply)
            # (4 unchanged attributes hidden)
        }
      ~ rules {
          ~ id           = "<snip>" -> (known after apply)
          ~ last_updated = "2023-08-18 16:36:15.176813 +0000 UTC" -> (known after apply)
          ~ ref          = "<snip>" -> (known after apply)
          ~ version      = "15" -> (known after apply)
            # (4 unchanged attributes hidden)

          ~ action_parameters {
              + version  = (known after apply)
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
      + rules {
          + action       = "log"
          + description  = "log traffic with headers"
          + enabled      = true
          + expression   = <<-EOT
                any(http.request.headers["custom-header"][*] matches ".?")
            EOT
          + id           = (known after apply)
          + last_updated = (known after apply)
          + ref          = (known after apply)
          + version      = (known after apply)
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

In 4.7.1 cloudflare provider

      ~ rules {
          ~ id           = "<snip>" -> (known after apply)
          + last_updated = (known after apply)
          ~ version      = "3" -> (known after apply)
            # (4 unchanged attributes hidden)
        }
      ~ rules {
          ~ id           = "<snip>" -> (known after apply)
          + last_updated = (known after apply)
          ~ version      = "3" -> (known after apply)
            # (4 unchanged attributes hidden)

          ~ action_parameters {
              + version  = (known after apply)
                # (1 unchanged attribute hidden)
            }

            # (1 unchanged block hidden)
        }
      + rules {
          + action       = "log"
          + description  = "log traffic with headers"
          + enabled      = true
          + expression   = <<-EOT
                any(http.request.headers["custom-header"][*] matches ".?")
            EOT
          + id           = (known after apply)
          + last_updated = (known after apply)
          + ref          = (known after apply)
          + version      = (known after apply)
        }

We also cannot set a lifecycle ignore rule because this does not work for us

  lifecycle {
    ignore_changes = [
      "rules.*.id",
      "rules.*.version",
      "rules.*.last_updated",
      "rules.*.ref",
      # "rules[*].id",
      # "rules.id",
    ]
  }

I'm told this might work but it's not scaleable. We have over 30 rules. I tried this and still see the drift!

  lifecycle {
    ignore_changes = [
      rules[0].id,
      rules[0].version,
      rules[0].last_updated,
      rules[0].ref,
      rules[1].id,
      rules[1].version,
      rules[1].last_updated,
      rules[1].ref,
      rules[2].id,
      rules[2].version,
      rules[2].last_updated,
      rules[2].ref,
      rules[3].id,
      rules[3].version,
      rules[3].last_updated,
      rules[3].ref,
      # ...
      rules[n].id,
      rules[n].version,
      rules[n].last_updated,
      rules[n].ref,
    ]
  }

We did not see this issue in 3.x cloudflare provider

I don't see any DiffSuppressFunc for any of the above drifted computed arguments. Unsure if the drift needs to be suppressed or if it needs to be investigated.

https://github.com/cloudflare/terraform-provider-cloudflare/blob/adbc0accaafe0df39e78fcfd61d092573e13a25d/internal/framework/service/rulesets/resource.go

References

Related issues

Related pr

resource/cloudflare_ruleset: fix various attributes #2511 released in 4.8.0

From change log

BREAKING CHANGES:

resource/cloudflare_ruleset: Prevent the rule ID, version and last updated attributes from being set (resource/cloudflare_ruleset: fix various attributes #2511)

The text was updated successfully, but these errors were encountered:

github-actions · 2023-08-16T22:46:08Z

Community Note

Voting for Prioritization

Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.

github-actions · 2023-08-16T22:46:31Z

Thank you for reporting this issue! For maintainers to dig into issues it is required that all issues include the entirety of TF_LOG=DEBUG output to be provided. The only parts that should be redacted are your user credentials in the X-Auth-Key, X-Auth-Email and Authorization HTTP headers. Details such as zone or account identifiers are not considered sensitive but can be redacted if you are very cautious. This log file provides additional context from Terraform, the provider and the Cloudflare API that helps in debugging issues. Without it, maintainers are very limited in what they can do and may hamper diagnosis efforts.

This issue has been marked with triage/needs-information and is unlikely to receive maintainer attention until the log file is provided making this a complete bug report.

ghost · 2023-08-19T06:26:36Z

I can provide this, but I'm not comfortable posting that here in a public forum. I'm an enterprise customer, can I create a case?

Rules, IPs, etc. would take a long while to redact.

Log uploaded to case id: #2901904

ghost · 2023-08-25T21:20:44Z

Thanks for the lifecycle hint. It can help with perma drift until a proper solution is identified.

github-actions · 2023-09-25T00:21:04Z

Marking this issue as stale due to 30 days of inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 7 days it will automatically be closed. Maintainers can also remove the lifecycle/stale label.
If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

DDuarte · 2023-10-10T08:16:01Z

Confirming this bug as well, any time there's a change to 1 rule, all the other rules in the ruleset get their mentioned fields updated in diff

ghost · 2023-11-01T17:12:55Z

@DDuarte and @nitrocode if you define a unique ref for the rules block, you should no longer see permanent drift.

Edit: Spoke too soon, the issue persists.

gabrielqs · 2023-11-14T15:08:51Z

+1 dealing with this issue.
given that we're migrating from multiple deprecated cloudflare_firewall_rules into a single new cloudflare_ruleset, this makes the plan outputs become really hard to read. currently using a list of 300 attributes being ignored by lifecycle rules 😅

vojkny · 2023-11-27T08:36:34Z

I tried to fix this something like:

    lifecycle {
      ignore_changes = ["rules[*].ref", "rules[*].last_updated"]
    }

However there is nothing like [*] in Terraform syntax, so this couldn't work.

nitrocode · 2023-11-28T05:31:29Z

@gabrielqs could you share what you use? I tried setting the ignore_changes and I could not get it to work.

github-actions · 2023-12-29T00:18:20Z

Marking this issue as stale due to 30 days of inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 7 days it will automatically be closed. Maintainers can also remove the lifecycle/stale label.
If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

gabrielqs · 2023-12-29T06:44:55Z

This is still a problem

gabrielqs · 2023-12-29T13:16:39Z

@nitrocode sorry, only seeing your message now. here's what we had to do to get around this:

  lifecycle {
    ignore_changes = [
      rules[0].id,
      rules[0].version,
      rules[0].last_updated,
      rules[0].ref,
      rules[0].action_parameters[0].version,
      rules[1].id,
      rules[1].version,
      rules[1].last_updated,
      rules[1].ref,
      rules[1].action_parameters[0].version,
      rules[2].id,
      rules[2].version,
      rules[2].last_updated,
      rules[2].ref,
      rules[2].action_parameters[0].version,
      ...

It is terrible, and we need to remind ourselves to add this every time we add a new rule, but at least the drift doesn't show up on every plan

nitrocode · 2023-12-30T16:29:05Z

I'm happy it works for you but that doesn't work for us. We still see the drift even with explicitly specifying each individual attributes per rule in ignore_changes.

Since these rules are all in a single terraform resource and we cannot get the plan in a reviewable state, we may have to consider moving these rules out of terraform and having them managed with a script that supplants all the rules upon pr merge.

cc @jacobbednarz friendly ping to get cloudflare's feedback here. This is the 2nd highest voted issue and ideally we should be able to manage and review changes within terraform without so much noise.

troymjones · 2024-01-22T17:57:12Z

I am providing a full TF_LOG=DEBUG for 2 scenarios, both of which are problematic when working with the Cloudflare ruleset resource. Please note that while there is a workaround (manually verifying them), this is a major issue for my team with a hundred rules in a zone and could result in a bad rule change making its way into the WAF.

Scenarios:

A rule is added in the middle of existing rules. Note that it looks like rules are drastically changing and a 'new' rule is being added to the bottom of the list. However, that 'new' rule is just an existing rule at the end. The rules are correctly being shown, but it is extremely confusing. When there are more than a couple rules, it becomes almost impossible to parse through the changes to make sure no mistakes were made. Also, all rules show changes to id, ref, version, etc, further complicating the issue. Link to gist
A rule is modified (ie name, expression, etc) but no other changes are made. This results in the plan showing all rules having changed. Again, if there are more than a couple rules, it makes it almost impossible to verify that the changes desired are what is shown in the plan. Link to gist

emilianoangieri · 2024-06-27T15:23:46Z

anyone got a resolution or a workaround for this?

ariretiarno · 2024-07-23T14:47:52Z

still an issue

YogitaDSurve · 2024-08-20T08:28:55Z

Has anyone came up with any workaround for this till it gets fixed by terraform?
It's been a year since this issue is raised 😕.

nitrocode · 2024-08-20T11:19:19Z

Has anyone tried using a separate terraform directory for these rules using the older 3.x provider ?

https://registry.terraform.io/providers/cloudflare/cloudflare/3.35.0/docs/resources/ruleset

This way you can use the 3.x only for the older ruleset resource and use the latest provider for all the other resources

github-actions · 2024-09-20T00:27:21Z

Marking this issue as stale due to 30 days of inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 7 days it will automatically be closed. Maintainers can also remove the lifecycle/stale label.
If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

troymjones · 2024-09-23T03:58:16Z

This is still an issue

ebachle · 2024-09-25T02:12:48Z

I'm willing to bet that the new major version of the provider fixes this based on some of what I read here. https://blog.cloudflare.com/automatically-generating-cloudflares-terraform-provider/

Haven't tested it but I likely will in the next few days.

github-actions · 2024-10-26T01:17:11Z

Marking this issue as stale due to 30 days of inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 7 days it will automatically be closed. Maintainers can also remove the lifecycle/stale label.
If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

ebachle · 2024-10-26T05:05:41Z

Go away stale bot

debu99 · 2024-10-30T04:10:54Z

got the same issue, but rules[xxx].action_parameters[0].version, doesn't work for me

github-actions · 2024-11-30T01:12:07Z

Marking this issue as stale due to 30 days of inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 7 days it will automatically be closed. Maintainers can also remove the lifecycle/stale label.
If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

ebachle · 2024-12-02T12:25:05Z

bump

Within the rulesets schema, we have a handful of fields that are `Computed`. The expectation of this directive is that it is a value not known at run time and may be provided by the remote. However, this premise will break down and not work correctly if the value is ever provided to the plan since the value is no longer "unknown". This subtle bug is one part of what is contributing to the additional output toil in #2690. To address this, I've removed the lines that modified these values and were being supplied to the plan operation. This takes the confusing and bloated output from looking like this: ``` Terraform will perform the following actions: # cloudflare_ruleset.rate_limiting_example will be updated in-place ~ resource "cloudflare_ruleset" "rate_limiting_example" { id = "87e1099f077f4e49bbfbe159217ff605" name = "restrict API requests count" # (4 unchanged attributes hidden) ~ rules { ~ id = "e3b97fdf354c4c24a7ac091c3537fbc5" -> (known after apply) ~ last_updated = "2024-01-31 04:02:55.651275 +0000 UTC" -> (known after apply) ~ ref = "e3b97fdf354c4c24a7ac091c3537fbc5" -> (known after apply) ~ version = "1" -> (known after apply) # (4 unchanged attributes hidden) # (1 unchanged block hidden) } + rules { + action = "block" + description = "rate limit for API" + enabled = true + expression = "(http.request.uri.path matches \"^/api0/\")" + id = (known after apply) + last_updated = (known after apply) + ref = (known after apply) + version = (known after apply) + ratelimit { + characteristics = [ + "cf.colo.id", + "ip.src", ] + mitigation_timeout = 600 + period = 60 + requests_per_period = 100 + requests_to_origin = false } } } Plan: 0 to add, 1 to change, 0 to destroy. ``` To a clearer and more concise output: ``` Terraform will perform the following actions: # cloudflare_ruleset.rate_limiting_example will be updated in-place ~ resource "cloudflare_ruleset" "rate_limiting_example" { id = "87e1099f077f4e49bbfbe159217ff605" name = "restrict API requests count" # (4 unchanged attributes hidden) ~ rules { id = "39ed6015367444e3905d838cadc1b9c2" + last_updated = (known after apply) + version = (known after apply) # (5 unchanged attributes hidden) # (1 unchanged block hidden) } + rules { + action = "block" + description = "rate limit for API" + enabled = true + expression = "(http.request.uri.path matches \"^/api0/\")" + id = (known after apply) + last_updated = (known after apply) + ref = (known after apply) + version = (known after apply) + ratelimit { + characteristics = [ + "cf.colo.id", + "ip.src", ] + mitigation_timeout = 600 + period = 60 + requests_per_period = 100 + requests_to_origin = false } } } Plan: 0 to add, 1 to change, 0 to destroy. ``` The second part of this confusing output is the `last_updated` and `version` output. The `last_updated` is pretty self explanatory however, there are some minor but important details about the version field here. Within ERE, it captures and is tied to a particular state of the ruleset rule at any point and is incremented when changes are detected to any parts of the rule. The nuance here is that ERE does not perform a diff of the current state and what is proposed. Instead, it autoincrements this field even if the payload is identical. So why is this important for the Terraform resource? Well, since we use explicit ordering via `ListNestedObject` schema attribute, we are sending all rules to the API even when only a single one changes to ensure we maintain the correctness the end user expects. Doing this causes the `last_updated` and `version` fields to always show as unknown values and result in updates. While the `last_updated` and `version` fields are useful in a context where you may reuse the `version`, value, it's not in Terraform. To remove that part of the diff, we're going to just remove them entirely from the schema. Given that users couldn't be using them in a configuration, we're going to release this within a minor version handling the cleanup internally to the provider. Supersedes #3091 Closes #2690

nitrocode · 2024-12-10T22:21:29Z

Sweet. Thank you @zakcutner for the PR and @jacobbednarz for merging and releasing. We're looking forward to using the latest version.

ebachle · 2024-12-11T02:27:18Z

Much appreciated guys! This will be awesome!

github-actions · 2024-12-11T04:58:52Z

This functionality has been released in v4.48.0 of the Terraform Cloudflare Provider.

Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

nitrocode added kind/bug Categorizes issue or PR as related to a bug. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Aug 16, 2023

github-actions bot added triage/needs-information Indicates an issue needs more information in order to work on it. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Aug 16, 2023

nitrocode changed the title ~~Drift in cloudflare_ruleset for http_request_firewall_custom phase~~ Drift in cloudflare_ruleset for http_request_firewall_custom phase for computed args id, last_updated, ref, version Aug 19, 2023

github-actions bot added the lifecycle/stale label Sep 25, 2023

This comment was marked as spam.

Sign in to view

github-actions bot removed the lifecycle/stale label Sep 26, 2023

This comment was marked as spam.

Sign in to view

github-actions bot added the lifecycle/stale label Dec 29, 2023

github-actions bot removed the lifecycle/stale label Dec 30, 2023

This comment was marked as spam.

Sign in to view

This comment has been minimized.

Sign in to view

Nmishin mentioned this issue Jan 27, 2024

Remove unneeded parameters from cloudflare_ruleset #3082

Closed

nitrocode mentioned this issue Jun 13, 2024

cloudflare_ruleset inline rules can change order of rules resulting in a difficult plan #3357

Closed

3 tasks

github-actions bot removed the lifecycle/stale label Jun 14, 2024

nitrocode mentioned this issue Aug 20, 2024

Support splat argument in lifecycle -> ignore_changes opentofu/opentofu#1933

Open

github-actions bot added the lifecycle/stale label Sep 20, 2024

github-actions bot removed the lifecycle/stale label Sep 24, 2024

github-actions bot added the lifecycle/stale label Oct 26, 2024

github-actions bot removed the lifecycle/stale label Oct 27, 2024

github-actions bot added the lifecycle/stale label Nov 30, 2024

github-actions bot removed the lifecycle/stale label Dec 3, 2024

This was referenced Dec 3, 2024

resource/cloudflare_ruleset: improve unknown value handling #4692

Closed

resource/cloudflare_ruleset: Improve diffs when only some rules are changed #4697

Merged

jacobbednarz closed this as completed in #4697 Dec 9, 2024

github-actions bot added this to the v4.48.0 milestone Dec 9, 2024

github-actions bot locked as resolved and limited conversation to collaborators Dec 11, 2024

Drift in cloudflare_ruleset for http_request_firewall_custom phase for computed args id, last_updated, ref, version #2690

Drift in cloudflare_ruleset for http_request_firewall_custom phase for computed args id, last_updated, ref, version #2690

Comments

nitrocode commented Aug 16, 2023 • edited Loading

Confirmation

Terraform and Cloudflare provider version

Affected resource(s)

Terraform configuration files

Link to debug output

Panic output

Expected output

Actual output

Steps to reproduce

Additional factoids

References

github-actions bot commented Aug 16, 2023

Community Note

github-actions bot commented Aug 16, 2023

ghost commented Aug 19, 2023 • edited by ghost Loading

ghost commented Aug 25, 2023

github-actions bot commented Sep 25, 2023

This comment was marked as spam.

This comment was marked as spam.

DDuarte commented Oct 10, 2023

This comment was marked as spam.

ghost commented Nov 1, 2023 • edited by ghost Loading

gabrielqs commented Nov 14, 2023

vojkny commented Nov 27, 2023

nitrocode commented Nov 28, 2023

github-actions bot commented Dec 29, 2023

gabrielqs commented Dec 29, 2023

gabrielqs commented Dec 29, 2023

nitrocode commented Dec 30, 2023

troymjones commented Jan 22, 2024

This comment was marked as spam.

This comment has been minimized.

emilianoangieri commented Jun 27, 2024

ariretiarno commented Jul 23, 2024

YogitaDSurve commented Aug 20, 2024

nitrocode commented Aug 20, 2024 • edited Loading

github-actions bot commented Sep 20, 2024

troymjones commented Sep 23, 2024

ebachle commented Sep 25, 2024

github-actions bot commented Oct 26, 2024

ebachle commented Oct 26, 2024

debu99 commented Oct 30, 2024

github-actions bot commented Nov 30, 2024

ebachle commented Dec 2, 2024

nitrocode commented Dec 10, 2024

ebachle commented Dec 11, 2024

github-actions bot commented Dec 11, 2024

Drift in `cloudflare_ruleset` for `http_request_firewall_custom` phase for computed args `id`, `last_updated`, `ref`, `version` #2690

Drift in `cloudflare_ruleset` for `http_request_firewall_custom` phase for computed args `id`, `last_updated`, `ref`, `version` #2690

nitrocode commented Aug 16, 2023 •

edited

Loading

ghost commented Aug 19, 2023 •

edited by ghost

Loading

ghost commented Nov 1, 2023 •

edited by ghost

Loading

nitrocode commented Aug 20, 2024 •

edited

Loading