Inconsistent result when creating cloudstack_firewall

[mwaag]

Hi,
this is my first try to open an issue here, please bear with me, if this is not the correct way. I will thankfully accept any hints to optimize it in the future. I will try to orientate to the issue-template from the cloudstack-project.
I may repeat this with different tf-versions / os'es. Let me know, if you need more information.

PROVIDER INFORMATION

• Versoin: 0.5.0

TERRAFORM INFORMATION

• Versoin: 1.8.2

HYPERVISOR INFORMATION

• Hypervisor: VMWare ESXi
• Version: 7.0.3

CLOUDSTACK VERSION

• 4.17.2.0

CONFIGURATION

main.tf:
terraform {
  required_providers {
    cloudstack = {
        source = "cloudstack/cloudstack"
        version = "0.5.0"
    }
  }
}

provider "cloudstack" {
  # Configuration Options
  api_url    = "${var.cloudstack_api_url}"
  api_key    = "${var.cloudstack_api_key}"
  secret_key = "${var.cloudstack_secret_key}"
}

resource "cloudstack_template" "ubuntu2204" {
  name = "Ubuntu 22.04"
  format = "OVA"
  hypervisor = "VMware"
  os_type = "Other Linux (64-bit)"
  url = "${var.cloudstack_template_url}"
  zone = "Enterprise"
  project = "mwatest01"
  is_dynamically_scalable = true
  is_extractable = false
  is_featured = false
  is_public = true
  password_enabled = false
  is_ready_timeout = 600
  
}

resource "cloudstack_network" "snw-demo" {
    name                = "demo-network"
    display_text        = "demo-network"
    cidr                = "172.16.0.0/24"
    network_offering    = "DefaultIsolatedNetworkOfferingWithSourceNatService"
    zone                = "Enterprise"
    project             = "mwatest01"
    source_nat_ip = true
  
}

# resource "cloudstack_ipaddress" "default-ip0" {
#   network_id = cloudstack_network.snw-demo.id
#   zone = "Enterprise"
#   project = "mwatest01"
#   
# }

resource "cloudstack_instance" "instance-demo" {
    name = "vm-demo"
    display_name = "vm-demo"
    service_offering = "XS Instanz"
    template = cloudstack_template.ubuntu2204.id
    project = "mwatest01"
    zone = "Enterprise"
    start_vm = true
    expunge = true
    network_id = cloudstack_network.snw-demo.id
  
}

resource "cloudstack_firewall" "default" {
  ip_address_id = cloudstack_network.snw-demo.source_nat_ip_id

  rule {
    cidr_list = ["172.26.251.57/32"]
    protocol = "tcp"
    ports = ["22"]
  }
}

OS / ENVIRONMENT

• OS: Ubuntu 22.04.3 LTS on Windows Subsystem for Linux (WSL 2) on Windows 11 (64-bit)

SUMMARY

Error: Provider produced inconsistent result after apply

when trying to deploy a firewall-rule to a simple isolated guest-network via cloudstack-provider.

DETAILS

• We utilize projects for client-isolation
• Service-offering for vm is a custom one
• Result is reproducable with newly assigned non-snat-ip-adress (see resource cloudstack_ipadress)

STEPS TO REPRODUCE

• create main.tf with contents mentioned above
• run terraform apply

EXPECTED RESULTS

• Rule will be created successfully
• Rule will be included in terraform state

ACTUAL RESULTS

• Rule is created successfully

• Rule is not included in terraform state (Rerun will try to create a new rule)

• Full Output:

terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # cloudstack_firewall.default will be created
  + resource "cloudstack_firewall" "default" {
      + id            = (known after apply)
      + ip_address_id = (known after apply)
      + managed       = false
      + parallelism   = 2

      + rule {
          + cidr_list = [
              + "172.26.251.57/32",
            ]
          + icmp_code = (known after apply)
          + icmp_type = (known after apply)
          + ports     = [
              + "22",
            ]
          + protocol  = "tcp"
          + uuids     = (known after apply)
        }
    }

  # cloudstack_instance.instance-demo will be created
  + resource "cloudstack_instance" "instance-demo" {
      + display_name     = "vm-demo"
      + expunge          = true
      + group            = (known after apply)
      + id               = (known after apply)
      + ip_address       = (known after apply)
      + name             = "vm-demo"
      + network_id       = (known after apply)
      + project          = "mwatest01"
      + root_disk_size   = (known after apply)
      + service_offering = "XS Instanz"
      + start_vm         = true
      + tags             = (known after apply)
      + template         = (known after apply)
      + uefi             = false
      + zone             = "Enterprise"
    }

  # cloudstack_network.snw-demo will be created
  + resource "cloudstack_network" "snw-demo" {
      + acl_id                = "none"
      + cidr                  = "172.16.0.0/24"
      + display_text          = "demo-network"
      + endip                 = (known after apply)
      + gateway               = (known after apply)
      + id                    = (known after apply)
      + name                  = "demo-network"
      + network_domain        = (known after apply)
      + network_offering      = "DefaultIsolatedNetworkOfferingWithSourceNatService"
      + project               = "mwatest01"
      + source_nat_ip         = true
      + source_nat_ip_address = (known after apply)
      + source_nat_ip_id      = (known after apply)
      + startip               = (known after apply)
      + tags                  = (known after apply)
      + zone                  = "Enterprise"
    }

  # cloudstack_template.ubuntu2204 will be created
  + resource "cloudstack_template" "ubuntu2204" {
      + display_text            = (known after apply)
      + format                  = "OVA"
      + hypervisor              = "VMware"
      + id                      = (known after apply)
      + is_dynamically_scalable = true
      + is_extractable          = false
      + is_featured             = false
      + is_public               = true
      + is_ready                = (known after apply)
      + is_ready_timeout        = 600
      + name                    = "Ubuntu 22.04"
      + os_type                 = "Other Linux (64-bit)"
      + password_enabled        = false
      + project                 = "mwatest01"
      + tags                    = (known after apply)
      + url                     = "http://20.82.104.206/Ubuntu_22.04_jammy_user.ova"
      + zone                    = "Enterprise"
    }

Plan: 4 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

cloudstack_network.snw-demo: Creating...
cloudstack_template.ubuntu2204: Creating...
cloudstack_network.snw-demo: Creation complete after 2s [id=68247c09-802a-44b6-bc59-e31a9230c7d1]
cloudstack_firewall.default: Creating...
cloudstack_template.ubuntu2204: Still creating... [10s elapsed]
cloudstack_template.ubuntu2204: Still creating... [20s elapsed]
cloudstack_template.ubuntu2204: Still creating... [30s elapsed]
cloudstack_template.ubuntu2204: Still creating... [40s elapsed]
cloudstack_template.ubuntu2204: Still creating... [50s elapsed]
cloudstack_template.ubuntu2204: Still creating... [1m0s elapsed]
cloudstack_template.ubuntu2204: Still creating... [1m10s elapsed]
cloudstack_template.ubuntu2204: Still creating... [1m20s elapsed]
cloudstack_template.ubuntu2204: Still creating... [1m30s elapsed]
cloudstack_template.ubuntu2204: Still creating... [1m40s elapsed]
cloudstack_template.ubuntu2204: Still creating... [1m50s elapsed]
cloudstack_template.ubuntu2204: Still creating... [2m0s elapsed]
cloudstack_template.ubuntu2204: Still creating... [2m10s elapsed]
cloudstack_template.ubuntu2204: Creation complete after 2m14s [id=5bd0e49f-942c-4893-853e-ab74d725d1fe]
cloudstack_instance.instance-demo: Creating...
cloudstack_instance.instance-demo: Still creating... [10s elapsed]
cloudstack_instance.instance-demo: Still creating... [20s elapsed]
cloudstack_instance.instance-demo: Still creating... [30s elapsed]
cloudstack_instance.instance-demo: Still creating... [40s elapsed]
cloudstack_instance.instance-demo: Still creating... [50s elapsed]
cloudstack_instance.instance-demo: Still creating... [1m0s elapsed]
cloudstack_instance.instance-demo: Still creating... [1m10s elapsed]
cloudstack_instance.instance-demo: Still creating... [1m20s elapsed]
cloudstack_instance.instance-demo: Still creating... [1m30s elapsed]
cloudstack_instance.instance-demo: Still creating... [1m40s elapsed]
cloudstack_instance.instance-demo: Still creating... [1m50s elapsed]
cloudstack_instance.instance-demo: Still creating... [2m0s elapsed]
cloudstack_instance.instance-demo: Still creating... [2m10s elapsed]
cloudstack_instance.instance-demo: Still creating... [2m21s elapsed]
cloudstack_instance.instance-demo: Still creating... [2m31s elapsed]
cloudstack_instance.instance-demo: Still creating... [2m41s elapsed]
cloudstack_instance.instance-demo: Creation complete after 2m48s [id=5f1e1fbc-a259-4a3a-bd27-0c2b2ecc2c56]
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to cloudstack_firewall.default, provider "provider[\"registry.terraform.io/cloudstack/cloudstack\"]" produced an unexpected new value: Root
│ object was present, but now absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

[mwaag]

My colleague just found this: https://www.reddit.com/r/Terraform/comments/m5nv14/comment/gr29zct/?utm_source=share&utm_medium=web2x&context=3

Using the argument 'managed = true' workaround my problem, but it seems it would be cleaner when the problem is catched from the provider. What do you think?

[kiranchavala]

@mwaag

Thanks for reporting the issue

The issue is occurring only if project are used.

Marking it as a bug and improvement request