Switch to using pd-balanced for all user & dask nodes

[yuvipanda]

We were using standard disk to save costs, but that brings with
it much slower node startup time, as images being pulled take time.
pd-balanced is a newer alternative to pure SSD disks that is
not as expensive, but provides much better performance than pd-standard.

I think the extra cost is worth the performance on all these
cases.

This was based on feedback on new node spinup performance in
#991 (comment)

[yuvipanda]

terraform apply for pilot hubs

�[0m�[32mSwitched to workspace "pilot-hubs".�[0m
google_artifact_registry_repository.registry: Refreshing state... [id=projects/two-eye-two-see/locations/us-central1/repositories/pilot-hubs-registry]
google_service_account.cluster_sa: Refreshing state... [id=projects/two-eye-two-see/serviceAccounts/[email protected]]
google_service_account.cd_sa: Refreshing state... [id=projects/two-eye-two-see/serviceAccounts/[email protected]]
google_project_iam_custom_role.identify_project_role: Refreshing state... [id=projects/two-eye-two-see/roles/pilot_hubs_user_sa_role]
google_project_iam_member.cd_sa_roles["roles/artifactregistry.writer"]: Refreshing state... [id=two-eye-two-see/roles/artifactregistry.writer/serviceAccount:[email protected]]
google_project_iam_member.cd_sa_roles["roles/container.admin"]: Refreshing state... [id=two-eye-two-see/roles/container.admin/serviceaccount:[email protected]]
google_project_iam_member.cluster_sa_roles["roles/logging.logWriter"]: Refreshing state... [id=two-eye-two-see/roles/logging.logWriter/serviceaccount:[email protected]]
google_project_iam_member.cluster_sa_roles["roles/monitoring.metricWriter"]: Refreshing state... [id=two-eye-two-see/roles/monitoring.metricWriter/serviceAccount:[email protected]]
google_service_account_key.cd_sa: Refreshing state... [id=projects/two-eye-two-see/serviceAccounts/[email protected]/keys/ee245c75f6c3fa5a36ae59597b356f4c6da80334]
google_project_iam_member.cluster_sa_roles["roles/stackdriver.resourceMetadata.writer"]: Refreshing state... [id=two-eye-two-see/roles/stackdriver.resourceMetadata.writer/serviceAccount:[email protected]]
google_project_iam_member.cluster_sa_roles["roles/artifactregistry.reader"]: Refreshing state... [id=two-eye-two-see/roles/artifactregistry.reader/serviceAccount:[email protected]]
google_project_iam_member.cluster_sa_roles["roles/monitoring.viewer"]: Refreshing state... [id=two-eye-two-see/roles/monitoring.viewer/serviceAccount:[email protected]]
google_project_iam_member.identify_project_binding: Refreshing state... [id=two-eye-two-see/projects/two-eye-two-see/roles/pilot_hubs_user_sa_role/serviceAccount:[email protected]]
google_container_cluster.cluster: Refreshing state... [id=projects/two-eye-two-see/locations/us-central1-b/clusters/pilot-hubs-cluster]
google_container_node_pool.notebook["user"]: Refreshing state... [id=projects/two-eye-two-see/locations/us-central1-b/clusters/pilot-hubs-cluster/nodePools/nb-user]
google_container_node_pool.notebook["paleo"]: Refreshing state... [id=projects/two-eye-two-see/locations/us-central1-b/clusters/pilot-hubs-cluster/nodePools/nb-paleo]
google_container_node_pool.dask_worker["worker"]: Refreshing state... [id=projects/two-eye-two-see/locations/us-central1-b/clusters/pilot-hubs-cluster/nodePools/dask-worker]
google_container_node_pool.core: Refreshing state... [id=projects/two-eye-two-see/locations/us-central1-b/clusters/pilot-hubs-cluster/nodePools/core-pool]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the

last "terraform apply":
google_container_cluster.cluster has changed
~ resource "google_container_cluster" "cluster" {

id                          = "projects/two-eye-two-see/locations/us-central1-b/clusters/pilot-hubs-cluster"

name                        = "pilot-hubs-cluster"

# (26 unchanged attributes hidden)
  ~ node_pool {
      ~ initial_node_count          = 20 -> 0
      ~ instance_group_urls         = [
          - "https://www.googleapis.com/compute/v1/projects/two-eye-two-see/zones/us-central1-b/instanceGroupManagers/gke-pilot-hubs-cluster-nb-paleo-a6c38cd7-grp",
          + "https://www.googleapis.com/compute/v1/projects/two-eye-two-see/zones/us-central1-b/instanceGroupManagers/gke-pilot-hubs-cluster-nb-user-dabe793e-grp",
        ]
      ~ managed_instance_group_urls = [
          - "https://www.googleapis.com/compute/beta/projects/two-eye-two-see/zones/us-central1-b/instanceGroups/gke-pilot-hubs-cluster-nb-paleo-a6c38cd7-grp",
          + "https://www.googleapis.com/compute/beta/projects/two-eye-two-see/zones/us-central1-b/instanceGroups/gke-pilot-hubs-cluster-nb-user-dabe793e-grp",
        ]
      ~ name                        = "nb-paleo" -> "nb-user"
        # (4 unchanged attributes hidden)

      ~ autoscaling {
          ~ max_node_count = 100 -> 20
          ~ min_node_count = 20 -> 0
        }


      ~ node_config {
          ~ labels            = {
              - "2i2c.org/community"           = "paleo" -> null
                # (2 unchanged elements hidden)
            }
            tags              = []
            # (12 unchanged attributes hidden)


            # (2 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }
  + node_pool {
      + initial_node_count          = 10
      + instance_group_urls         = [
          + "https://www.googleapis.com/compute/v1/projects/two-eye-two-see/zones/us-central1-b/instanceGroupManagers/gke-pilot-hubs-cluster-nb-paleo-690ff684-grp",
        ]
      + managed_instance_group_urls = [
          + "https://www.googleapis.com/compute/beta/projects/two-eye-two-see/zones/us-central1-b/instanceGroups/gke-pilot-hubs-cluster-nb-paleo-690ff684-grp",
        ]
      + max_pods_per_node           = 0
      + name                        = "nb-paleo"
      + node_count                  = 1
      + node_locations              = [
          + "us-central1-b",
        ]
      + version                     = "1.20.12-gke.1500"

      + autoscaling {
          + max_node_count = 100
          + min_node_count = 0
        }

      + management {
          + auto_repair  = true
          + auto_upgrade = false
        }

      + node_config {
          + disk_size_gb      = 100
          + disk_type         = "pd-standard"
          + guest_accelerator = []
          + image_type        = "COS_CONTAINERD"
          + labels            = {
              + "2i2c.org/community"           = "paleo"
              + "hub.jupyter.org/node-purpose" = "user"
              + "k8s.dask.org/node-purpose"    = "scheduler"
            }
          + local_ssd_count   = 0
          + machine_type      = "n1-highmem-4"
          + metadata          = {
              + "disable-legacy-endpoints" = "true"
            }
          + oauth_scopes      = [
              + "https://www.googleapis.com/auth/cloud-platform",
            ]
          + preemptible       = false
          + service_account   = "[email protected]"
          + spot              = false
          + tags              = []
          + taint             = [
              + {
                  + effect = "NO_SCHEDULE"
                  + key    = "hub.jupyter.org_dedicated"
                  + value  = "user"
                },
            ]

          + shielded_instance_config {
              + enable_integrity_monitoring = true
              + enable_secure_boot          = false
            }

          + workload_metadata_config {
              + mode = "GKE_METADATA"
            }
        }

      + upgrade_settings {
          + max_surge       = 1
          + max_unavailable = 0
        }
    }




    # (16 unchanged blocks hidden)
}

google_container_node_pool.notebook["paleo"] has changed
~ resource "google_container_node_pool" "notebook" {

id                          = "projects/two-eye-two-see/locations/us-central1-b/clusters/pilot-hubs-cluster/nodePools/nb-paleo"

~ initial_node_count          = 0 -> 10

name                        = "nb-paleo"

~ node_count                  = 0 -> 1

# (7 unchanged attributes hidden)
  ~ autoscaling {
      ~ max_node_count = 20 -> 100
        # (1 unchanged attribute hidden)
    }



    # (3 unchanged blocks hidden)
}

google_project_iam_member.cd_sa_roles["roles/container.admin"] has changed
~ resource "google_project_iam_member" "cd_sa_roles" {

~ etag    = "BwXZxSY7xrc=" -> "BwXZxSuDQAQ="

id      = "two-eye-two-see/roles/container.admin/serviceaccount:[email protected]"

# (3 unchanged attributes hidden)

}
google_project_iam_member.cluster_sa_roles["roles/logging.logWriter"] has changed
~ resource "google_project_iam_member" "cluster_sa_roles" {

~ etag    = "BwXZxSY7xrc=" -> "BwXZxSuDQAQ="

id      = "two-eye-two-see/roles/logging.logWriter/serviceaccount:[email protected]"

# (3 unchanged attributes hidden)

}
Unless you have made equivalent changes to your configuration, or ignored the

relevant attributes using ignore_changes, the following plan may include

actions to undo or respond to these changes.
─────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution

plan. Resource actions are indicated with the following symbols:

-/+ destroy and then create replacement
Terraform will perform the following actions:
google_container_node_pool.dask_worker["worker"] must be replaced
-/+ resource "google_container_node_pool" "dask_worker" {

~ id                          = "projects/two-eye-two-see/locations/us-central1-b/clusters/pilot-hubs-cluster/nodePools/dask-worker" -> (known after apply)

~ instance_group_urls         = [

- "https://www.googleapis.com/compute/v1/projects/two-eye-two-see/zones/us-central1-b/instanceGroupManagers/gke-pilot-hubs-cluster-dask-worker-e1438a35-grp",

] -> (known after apply)

~ managed_instance_group_urls = [

- "https://www.googleapis.com/compute/v1/projects/two-eye-two-see/zones/us-central1-b/instanceGroups/gke-pilot-hubs-cluster-dask-worker-e1438a35-grp",

] -> (known after apply)

+ max_pods_per_node           = (known after apply)

name                        = "dask-worker"

+ name_prefix                 = (known after apply)

~ node_count                  = 0 -> (known after apply)

~ node_locations              = [

- "us-central1-b",

] -> (known after apply)

+ operation                   = (known after apply)

~ version                     = "1.19.12-gke.2100" -> (known after apply)

# (4 unchanged attributes hidden)
  ~ node_config {
      ~ disk_size_gb      = 100 -> (known after apply)
      ~ disk_type         = "pd-ssd" -> "pd-balanced" # forces replacement
      ~ guest_accelerator = [] -> (known after apply)
      ~ image_type        = "COS_CONTAINERD" -> (known after apply)
      ~ local_ssd_count   = 0 -> (known after apply)
      ~ metadata          = {
          - "disable-legacy-endpoints" = "true"
        } -> (known after apply)
        tags              = []
        # (6 unchanged attributes hidden)

      ~ shielded_instance_config {
          ~ enable_integrity_monitoring = true -> (known after apply)
          ~ enable_secure_boot          = false -> (known after apply)
        }

        # (1 unchanged block hidden)
    }

  ~ upgrade_settings {
      ~ max_surge       = 1 -> (known after apply)
      ~ max_unavailable = 0 -> (known after apply)
    }
    # (2 unchanged blocks hidden)
}

google_container_node_pool.notebook["paleo"] must be replaced
-/+ resource "google_container_node_pool" "notebook" {

~ id                          = "projects/two-eye-two-see/locations/us-central1-b/clusters/pilot-hubs-cluster/nodePools/nb-paleo" -> (known after apply)

~ initial_node_count          = 10 -> 0 # forces replacement

~ instance_group_urls         = [

- "https://www.googleapis.com/compute/v1/projects/two-eye-two-see/zones/us-central1-b/instanceGroupManagers/gke-pilot-hubs-cluster-nb-paleo-690ff684-grp",

] -> (known after apply)

~ managed_instance_group_urls = [

- "https://www.googleapis.com/compute/v1/projects/two-eye-two-see/zones/us-central1-b/instanceGroups/gke-pilot-hubs-cluster-nb-paleo-690ff684-grp",

] -> (known after apply)

+ max_pods_per_node           = (known after apply)

name                        = "nb-paleo"

+ name_prefix                 = (known after apply)

~ node_count                  = 1 -> (known after apply)

~ node_locations              = [

- "us-central1-b",

] -> (known after apply)

+ operation                   = (known after apply)

~ version                     = "1.20.12-gke.1500" -> (known after apply)

# (3 unchanged attributes hidden)
  ~ autoscaling {
      ~ max_node_count = 100 -> 20
        # (1 unchanged attribute hidden)
    }


  ~ node_config {
      ~ disk_size_gb      = 100 -> (known after apply)
      ~ disk_type         = "pd-standard" -> "pd-balanced" # forces replacement
      ~ guest_accelerator = [] -> (known after apply)
      ~ image_type        = "COS_CONTAINERD" -> (known after apply)
      ~ local_ssd_count   = 0 -> (known after apply)
      ~ metadata          = {
          - "disable-legacy-endpoints" = "true"
        } -> (known after apply)
        tags              = []
        # (6 unchanged attributes hidden)

      ~ shielded_instance_config {
          ~ enable_integrity_monitoring = true -> (known after apply)
          ~ enable_secure_boot          = false -> (known after apply)
        }

        # (1 unchanged block hidden)
    }

  ~ upgrade_settings {
      ~ max_surge       = 1 -> (known after apply)
      ~ max_unavailable = 0 -> (known after apply)
    }
    # (1 unchanged block hidden)
}

google_container_node_pool.notebook["user"] must be replaced
-/+ resource "google_container_node_pool" "notebook" {

~ id                          = "projects/two-eye-two-see/locations/us-central1-b/clusters/pilot-hubs-cluster/nodePools/nb-user" -> (known after apply)

~ instance_group_urls         = [

- "https://www.googleapis.com/compute/v1/projects/two-eye-two-see/zones/us-central1-b/instanceGroupManagers/gke-pilot-hubs-cluster-nb-user-dabe793e-grp",

] -> (known after apply)

~ managed_instance_group_urls = [

- "https://www.googleapis.com/compute/v1/projects/two-eye-two-see/zones/us-central1-b/instanceGroups/gke-pilot-hubs-cluster-nb-user-dabe793e-grp",

] -> (known after apply)

+ max_pods_per_node           = (known after apply)

name                        = "nb-user"

+ name_prefix                 = (known after apply)

~ node_count                  = 0 -> (known after apply)

~ node_locations              = [

- "us-central1-b",

] -> (known after apply)

+ operation                   = (known after apply)

~ version                     = "1.20.12-gke.1500" -> (known after apply)

# (4 unchanged attributes hidden)
  ~ node_config {
      ~ disk_size_gb      = 100 -> (known after apply)
      ~ disk_type         = "pd-standard" -> "pd-balanced" # forces replacement
      ~ guest_accelerator = [] -> (known after apply)
      ~ image_type        = "COS_CONTAINERD" -> (known after apply)
      ~ local_ssd_count   = 0 -> (known after apply)
      ~ metadata          = {
          - "disable-legacy-endpoints" = "true"
        } -> (known after apply)
        tags              = []
        # (6 unchanged attributes hidden)

      ~ shielded_instance_config {
          ~ enable_integrity_monitoring = true -> (known after apply)
          ~ enable_secure_boot          = false -> (known after apply)
        }

        # (1 unchanged block hidden)
    }

  ~ upgrade_settings {
      ~ max_surge       = 1 -> (known after apply)
      ~ max_unavailable = 0 -> (known after apply)
    }
    # (2 unchanged blocks hidden)
}

Plan: 3 to add, 0 to change, 3 to destroy.
Warning: Argument is deprecated
with google_filestore_instance.homedirs,

on storage.tf line 4, in resource "google_filestore_instance" "homedirs":

4:   zone = var.zone
Deprecated in favor of location.
─────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't

guarantee to take exactly these actions if you run "terraform apply" now.

[yuvipanda]

terraform apply for cloudbank

�[0m�[1mgoogle_artifact_registry_repository.registry: Refreshing state... [id=projects/cb-1003-1696/locations/us-central1/repositories/cb-registry]�[0m
�[0m�[1mgoogle_service_account.cd_sa: Refreshing state... [id=projects/cb-1003-1696/serviceAccounts/[email protected]]�[0m
�[0m�[1mgoogle_service_account.cluster_sa: Refreshing state... [id=projects/cb-1003-1696/serviceAccounts/[email protected]]�[0m
�[0m�[1mgoogle_project_iam_custom_role.identify_project_role: Refreshing state... [id=projects/cb-1003-1696/roles/cb_user_sa_role]�[0m
�[0m�[1mgoogle_project_iam_member.cd_sa_roles["roles/artifactregistry.writer"]: Refreshing state... [id=cb-1003-1696/roles/artifactregistry.writer/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.cd_sa_roles["roles/container.admin"]: Refreshing state... [id=cb-1003-1696/roles/container.admin/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_service_account_key.cd_sa: Refreshing state... [id=projects/cb-1003-1696/serviceAccounts/[email protected]/keys/fc50105d4d954fbe97e4453d0988df9dc88a762a]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/logging.logWriter"]: Refreshing state... [id=cb-1003-1696/roles/logging.logWriter/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/artifactregistry.reader"]: Refreshing state... [id=cb-1003-1696/roles/artifactregistry.reader/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/monitoring.metricWriter"]: Refreshing state... [id=cb-1003-1696/roles/monitoring.metricWriter/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/monitoring.viewer"]: Refreshing state... [id=cb-1003-1696/roles/monitoring.viewer/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/stackdriver.resourceMetadata.writer"]: Refreshing state... [id=cb-1003-1696/roles/stackdriver.resourceMetadata.writer/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.identify_project_binding: Refreshing state... [id=cb-1003-1696/projects/cb-1003-1696/roles/cb_user_sa_role/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_container_cluster.cluster: Refreshing state... [id=projects/cb-1003-1696/locations/us-central1-b/clusters/cb-cluster]�[0m
�[0m�[1mgoogle_container_node_pool.dask_worker["worker"]: Refreshing state... [id=projects/cb-1003-1696/locations/us-central1-b/clusters/cb-cluster/nodePools/dask-worker]�[0m
�[0m�[1mgoogle_container_node_pool.notebook["user"]: Refreshing state... [id=projects/cb-1003-1696/locations/us-central1-b/clusters/cb-cluster/nodePools/nb-user]�[0m
�[0m�[1mgoogle_container_node_pool.core: Refreshing state... [id=projects/cb-1003-1696/locations/us-central1-b/clusters/cb-cluster/nodePools/core-pool]�[0m
�[0m
�[1m�[36mNote:�[0m�[1m Objects have changed outside of Terraform�[0m
�[0mTerraform detected the following changes made outside of Terraform since the

last "terraform apply":
�[1m  # google_container_cluster.cluster�[0m has changed�[0m�[0m

�[0m  �[33m~�[0m�[0m resource "google_container_cluster" "cluster" {

�[1m�[0mid�[0m�[0m                          = "projects/cb-1003-1696/locations/us-central1-b/clusters/cb-cluster"

�[1m�[0mname�[0m�[0m                        = "cb-cluster"

�[90m# (26 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_pool {
        �[1m�[0mname�[0m�[0m                        = "nb-user"
      �[33m~�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 2 �[33m->�[0m �[0m1
        �[90m# (6 unchanged attributes hidden)�[0m�[0m




        �[90m# (4 unchanged blocks hidden)�[0m�[0m
    }




    �[90m# (16 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["user"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_container_node_pool" "notebook" {

�[1m�[0mid�[0m�[0m                          = "projects/cb-1003-1696/locations/us-central1-b/clusters/cb-cluster/nodePools/nb-user"

�[1m�[0mname�[0m�[0m                        = "nb-user"

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 2 �[33m->�[0m �[0m1

�[90m# (8 unchanged attributes hidden)�[0m�[0m
    �[90m# (4 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_project_iam_member.cd_sa_roles["roles/artifactregistry.writer"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_project_iam_member" "cd_sa_roles" {

�[33m�[0m �[0m�[1m�[0metag�[0m�[0m    = "BwXJT6xb3i4=" �[33m->�[0m �[0m"BwXZqWSXAKE="

�[1m�[0mid�[0m�[0m      = "cb-1003-1696/roles/artifactregistry.writer/serviceaccount:[email protected]"

�[90m# (3 unchanged attributes hidden)�[0m�[0m

}
�[1m  # google_project_iam_member.cd_sa_roles["roles/container.admin"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_project_iam_member" "cd_sa_roles" {

�[33m�[0m �[0m�[1m�[0metag�[0m�[0m    = "BwXJT6xb3i4=" �[33m->�[0m �[0m"BwXZqWSXAKE="

�[1m�[0mid�[0m�[0m      = "cb-1003-1696/roles/container.admin/serviceaccount:[email protected]"

�[90m# (3 unchanged attributes hidden)�[0m�[0m

}
�[1m  # google_project_iam_member.cluster_sa_roles["roles/artifactregistry.reader"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_project_iam_member" "cluster_sa_roles" {

�[33m�[0m �[0m�[1m�[0metag�[0m�[0m    = "BwXJT6xb3i4=" �[33m->�[0m �[0m"BwXZqWSXAKE="

�[1m�[0mid�[0m�[0m      = "cb-1003-1696/roles/artifactregistry.reader/serviceaccount:[email protected]"

�[90m# (3 unchanged attributes hidden)�[0m�[0m

}
�[1m  # google_project_iam_member.cluster_sa_roles["roles/logging.logWriter"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_project_iam_member" "cluster_sa_roles" {

�[33m�[0m �[0m�[1m�[0metag�[0m�[0m    = "BwXJT6xb3i4=" �[33m->�[0m �[0m"BwXZqWSXAKE="

�[1m�[0mid�[0m�[0m      = "cb-1003-1696/roles/logging.logWriter/serviceaccount:[email protected]"

�[90m# (3 unchanged attributes hidden)�[0m�[0m

}
�[1m  # google_project_iam_member.cluster_sa_roles["roles/monitoring.metricWriter"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_project_iam_member" "cluster_sa_roles" {

�[33m�[0m �[0m�[1m�[0metag�[0m�[0m    = "BwXJT6xb3i4=" �[33m->�[0m �[0m"BwXZqWSXAKE="

�[1m�[0mid�[0m�[0m      = "cb-1003-1696/roles/monitoring.metricWriter/serviceaccount:[email protected]"

�[90m# (3 unchanged attributes hidden)�[0m�[0m

}
�[1m  # google_project_iam_member.cluster_sa_roles["roles/monitoring.viewer"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_project_iam_member" "cluster_sa_roles" {

�[33m�[0m �[0m�[1m�[0metag�[0m�[0m    = "BwXJT6xb3i4=" �[33m->�[0m �[0m"BwXZqWSXAKE="

�[1m�[0mid�[0m�[0m      = "cb-1003-1696/roles/monitoring.viewer/serviceaccount:[email protected]"

�[90m# (3 unchanged attributes hidden)�[0m�[0m

}
�[1m  # google_project_iam_member.cluster_sa_roles["roles/stackdriver.resourceMetadata.writer"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_project_iam_member" "cluster_sa_roles" {

�[33m�[0m �[0m�[1m�[0metag�[0m�[0m    = "BwXJT6xb3i4=" �[33m->�[0m �[0m"BwXZqWSXAKE="

�[1m�[0mid�[0m�[0m      = "cb-1003-1696/roles/stackdriver.resourceMetadata.writer/serviceaccount:[email protected]"

�[90m# (3 unchanged attributes hidden)�[0m�[0m

}
�[1m  # google_project_iam_member.identify_project_binding�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_project_iam_member" "identify_project_binding" {

�[33m�[0m �[0m�[1m�[0metag�[0m�[0m    = "BwXJT6xb3i4=" �[33m->�[0m �[0m"BwXZqWSXAKE="

�[1m�[0mid�[0m�[0m      = "cb-1003-1696/projects/cb-1003-1696/roles/cb_user_sa_role/serviceaccount:[email protected]"

�[90m# (3 unchanged attributes hidden)�[0m�[0m

}
Unless you have made equivalent changes to your configuration, or ignored the

relevant attributes using ignore_changes, the following plan may include

actions to undo or respond to these changes.

�[90m

─────────────────────────────────────────────────────────────────────────────�[0m
Terraform used the selected providers to generate the following execution

plan. Resource actions are indicated with the following symbols:

�[31m-�[0m/�[32m+�[0m destroy and then create replacement

�[0m

Terraform will perform the following actions:
�[1m  # google_container_node_pool.dask_worker["worker"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "dask_worker" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/cb-1003-1696/locations/us-central1-b/clusters/cb-cluster/nodePools/dask-worker" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/cb-1003-1696/zones/us-central1-b/instanceGroupManagers/gke-cb-cluster-dask-worker-2894ec13-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/cb-1003-1696/zones/us-central1-b/instanceGroups/gke-cb-cluster-dask-worker-2894ec13-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "dask-worker"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2100" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-ssd" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["user"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "notebook" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/cb-1003-1696/locations/us-central1-b/clusters/cb-cluster/nodePools/nb-user" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/cb-1003-1696/zones/us-central1-b/instanceGroupManagers/gke-cb-cluster-nb-user-05ebb8d0-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/cb-1003-1696/zones/us-central1-b/instanceGroups/gke-cb-cluster-nb-user-05ebb8d0-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "nb-user"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 1 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2100" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-standard" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[0m�[1mPlan:�[0m 2 to add, 0 to change, 2 to destroy.

�[0m�[33m╷�[0m�[0m

�[33m│�[0m �[0m�[1m�[33mWarning: �[0m�[0m�[1mArgument is deprecated�[0m

�[33m│�[0m �[0m

�[33m│�[0m �[0m�[0m  with google_filestore_instance.homedirs,

�[33m│�[0m �[0m  on storage.tf line 4, in resource "google_filestore_instance" "homedirs":

�[33m│�[0m �[0m   4:   zone = �[4mvar.zone�[0m�[0m

�[33m│�[0m �[0m

�[33m│�[0m �[0mDeprecated in favor of location.

�[33m╵�[0m�[0m

�[90m

─────────────────────────────────────────────────────────────────────────────�[0m
Note: You didn't use the -out option to save this plan, so Terraform can't

guarantee to take exactly these actions if you run "terraform apply" now.

[yuvipanda]

terraform apply for meom-ige:

�[0m�[1mgoogle_service_account.cd_sa: Refreshing state... [id=projects/meom-ige-cnrs/serviceAccounts/[email protected]]�[0m
�[0m�[1mgoogle_project_iam_custom_role.identify_project_role: Refreshing state... [id=projects/meom-ige-cnrs/roles/meom_ige_user_sa_role]�[0m
�[0m�[1mgoogle_service_account.cluster_sa: Refreshing state... [id=projects/meom-ige-cnrs/serviceAccounts/[email protected]]�[0m
�[0m�[1mgoogle_storage_bucket.user_buckets["data"]: Refreshing state... [id=meom-ige-data]�[0m
�[0m�[1mgoogle_storage_bucket.user_buckets["scratch"]: Refreshing state... [id=meom-ige-scratch]�[0m
�[0m�[1mgoogle_artifact_registry_repository.registry: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1/repositories/meom-ige-registry]�[0m
�[0m�[1mgoogle_service_account_key.cd_sa: Refreshing state... [id=projects/meom-ige-cnrs/serviceAccounts/[email protected]/keys/6fb3ca469f000201aa0e2ea24ff02515324b1ffe]�[0m
�[0m�[1mgoogle_project_iam_member.cd_sa_roles["roles/artifactregistry.writer"]: Refreshing state... [id=meom-ige-cnrs/roles/artifactregistry.writer/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.cd_sa_roles["roles/container.admin"]: Refreshing state... [id=meom-ige-cnrs/roles/container.admin/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_storage_bucket_iam_member.member["data"]: Refreshing state... [id=b/meom-ige-data/roles/storage.admin/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/stackdriver.resourceMetadata.writer"]: Refreshing state... [id=meom-ige-cnrs/roles/stackdriver.resourceMetadata.writer/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_storage_bucket_iam_member.member["scratch"]: Refreshing state... [id=b/meom-ige-scratch/roles/storage.admin/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/artifactregistry.reader"]: Refreshing state... [id=meom-ige-cnrs/roles/artifactregistry.reader/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/monitoring.metricWriter"]: Refreshing state... [id=meom-ige-cnrs/roles/monitoring.metricWriter/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/monitoring.viewer"]: Refreshing state... [id=meom-ige-cnrs/roles/monitoring.viewer/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_container_cluster.cluster: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/logging.logWriter"]: Refreshing state... [id=meom-ige-cnrs/roles/logging.logWriter/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_project_iam_member.identify_project_binding: Refreshing state... [id=meom-ige-cnrs/projects/meom-ige-cnrs/roles/meom_ige_user_sa_role/serviceaccount:[email protected]]�[0m
�[0m�[1mgoogle_container_node_pool.dask_worker["small"]: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/dask-small]�[0m
�[0m�[1mgoogle_container_node_pool.notebook["huge"]: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-huge]�[0m
�[0m�[1mgoogle_container_node_pool.core: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/core-pool]�[0m
�[0m�[1mgoogle_container_node_pool.notebook["small"]: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-small]�[0m
�[0m�[1mgoogle_container_node_pool.dask_worker["very-large"]: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/dask-very-large]�[0m
�[0m�[1mgoogle_container_node_pool.notebook["large"]: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-large]�[0m
�[0m�[1mgoogle_container_node_pool.dask_worker["medium"]: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/dask-medium]�[0m
�[0m�[1mgoogle_container_node_pool.notebook["medium"]: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-medium]�[0m
�[0m�[1mgoogle_container_node_pool.dask_worker["huge"]: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/dask-huge]�[0m
�[0m�[1mgoogle_container_node_pool.notebook["very-large"]: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-very-large]�[0m
�[0m�[1mgoogle_container_node_pool.dask_worker["large"]: Refreshing state... [id=projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/dask-large]�[0m
�[0m
�[1m�[36mNote:�[0m�[1m Objects have changed outside of Terraform�[0m
�[0mTerraform detected the following changes made outside of Terraform since the

last "terraform apply":
�[1m  # google_container_cluster.cluster�[0m has changed�[0m�[0m

�[0m  �[33m~�[0m�[0m resource "google_container_cluster" "cluster" {

�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster"

�[1m�[0mname�[0m�[0m                        = "meom-ige-cluster"

�[90m# (26 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_pool {
        �[1m�[0mname�[0m�[0m                        = "nb-medium"
      �[33m~�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m1
        �[90m# (6 unchanged attributes hidden)�[0m�[0m




        �[90m# (4 unchanged blocks hidden)�[0m�[0m
    }




    �[90m# (24 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["medium"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_container_node_pool" "notebook" {

�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-medium"

�[1m�[0mname�[0m�[0m                        = "nb-medium"

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m1

�[90m# (8 unchanged attributes hidden)�[0m�[0m
    �[90m# (4 unchanged blocks hidden)�[0m�[0m
}

Unless you have made equivalent changes to your configuration, or ignored the

relevant attributes using ignore_changes, the following plan may include

actions to undo or respond to these changes.

�[90m

─────────────────────────────────────────────────────────────────────────────�[0m
Terraform used the selected providers to generate the following execution

plan. Resource actions are indicated with the following symbols:

�[31m-�[0m/�[32m+�[0m destroy and then create replacement

�[0m

Terraform will perform the following actions:
�[1m  # google_container_node_pool.dask_worker["huge"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "dask_worker" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/dask-huge" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroupManagers/gke-meom-ige-cluster-dask-huge-f0c9fcd8-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroups/gke-meom-ige-cluster-dask-huge-f0c9fcd8-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "dask-huge"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-ssd" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.dask_worker["large"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "dask_worker" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/dask-large" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroupManagers/gke-meom-ige-cluster-dask-large-5103975d-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroups/gke-meom-ige-cluster-dask-large-5103975d-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "dask-large"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-ssd" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.dask_worker["medium"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "dask_worker" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/dask-medium" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroupManagers/gke-meom-ige-cluster-dask-medium-c97debc9-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroups/gke-meom-ige-cluster-dask-medium-c97debc9-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "dask-medium"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-ssd" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.dask_worker["small"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "dask_worker" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/dask-small" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroupManagers/gke-meom-ige-cluster-dask-small-7ffa49d9-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroups/gke-meom-ige-cluster-dask-small-7ffa49d9-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "dask-small"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-ssd" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.dask_worker["very-large"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "dask_worker" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/dask-very-large" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroupManagers/gke-meom-ige-cluster-dask-very-large-91594b39-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroups/gke-meom-ige-cluster-dask-very-large-91594b39-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "dask-very-large"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-ssd" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["huge"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "notebook" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-huge" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroupManagers/gke-meom-ige-cluster-nb-huge-14f7c500-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroups/gke-meom-ige-cluster-nb-huge-14f7c500-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "nb-huge"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-standard" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["large"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "notebook" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-large" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroupManagers/gke-meom-ige-cluster-nb-large-52a70d65-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroups/gke-meom-ige-cluster-nb-large-52a70d65-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "nb-large"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-standard" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["medium"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "notebook" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-medium" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroupManagers/gke-meom-ige-cluster-nb-medium-fc542ba4-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroups/gke-meom-ige-cluster-nb-medium-fc542ba4-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "nb-medium"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 1 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-standard" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["small"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "notebook" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-small" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroupManagers/gke-meom-ige-cluster-nb-small-52f9c02f-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroups/gke-meom-ige-cluster-nb-small-52f9c02f-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "nb-small"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-standard" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["very-large"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "notebook" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/meom-ige-cnrs/locations/us-central1-b/clusters/meom-ige-cluster/nodePools/nb-very-large" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroupManagers/gke-meom-ige-cluster-nb-very-large-b79ad0a7-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/meom-ige-cnrs/zones/us-central1-b/instanceGroups/gke-meom-ige-cluster-nb-very-large-b79ad0a7-grp",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = (known after apply)

�[1m�[0mname�[0m�[0m                        = "nb-very-large"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.19.12-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-standard" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

      �[33m~�[0m �[0mworkload_metadata_config {
          �[32m+�[0m �[0m�[1m�[0mmode�[0m�[0m = "MODE_UNSPECIFIED"
        }
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[0m�[1mPlan:�[0m 10 to add, 0 to change, 10 to destroy.

�[0m�[33m╷�[0m�[0m

�[33m│�[0m �[0m�[1m�[33mWarning: �[0m�[0m�[1mArgument is deprecated�[0m

�[33m│�[0m �[0m

�[33m│�[0m �[0m�[0m  with google_filestore_instance.homedirs,

�[33m│�[0m �[0m  on storage.tf line 4, in resource "google_filestore_instance" "homedirs":

�[33m│�[0m �[0m   4:   zone = �[4mvar.zone�[0m�[0m

�[33m│�[0m �[0m

�[33m│�[0m �[0mDeprecated in favor of location.

�[33m╵�[0m�[0m

�[90m

─────────────────────────────────────────────────────────────────────────────�[0m
Note: You didn't use the -out option to save this plan, so Terraform can't

guarantee to take exactly these actions if you run "terraform apply" now.

[yuvipanda]

terraform plan output for pangeo-hubs.

�[0m�[1mgoogle_artifact_registry_repository.registry: Refreshing state... [id=projects/pangeo-integration-te-3eea/locations/us-central1/repositories/pangeo-hubs-registry]�[0m
�[0m�[1mgoogle_project_iam_custom_role.identify_project_role: Refreshing state... [id=projects/pangeo-integration-te-3eea/roles/pangeo_hubs_user_sa_role]�[0m
�[0m�[1mgoogle_service_account.cd_sa: Refreshing state... [id=projects/pangeo-integration-te-3eea/serviceAccounts/pangeo-hubs-cd-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_service_account.cluster_sa: Refreshing state... [id=projects/pangeo-integration-te-3eea/serviceAccounts/pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_storage_bucket.user_buckets["pangeo-scratch"]: Refreshing state... [id=pangeo-hubs-pangeo-scratch]�[0m
�[0m�[1mgoogle_service_account_key.cd_sa: Refreshing state... [id=projects/pangeo-integration-te-3eea/serviceAccounts/pangeo-hubs-cd-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com/keys/a49552f97c52549eff834dd54e80865715c0b953]�[0m
�[0m�[1mgoogle_storage_bucket_iam_member.member["pangeo-scratch"]: Refreshing state... [id=b/pangeo-hubs-pangeo-scratch/roles/storage.admin/serviceAccount:pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_project_iam_member.cd_sa_roles["roles/container.admin"]: Refreshing state... [id=pangeo-integration-te-3eea/roles/container.admin/serviceAccount:pangeo-hubs-cd-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_project_iam_member.cd_sa_roles["roles/artifactregistry.writer"]: Refreshing state... [id=pangeo-integration-te-3eea/roles/artifactregistry.writer/serviceAccount:pangeo-hubs-cd-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/monitoring.viewer"]: Refreshing state... [id=pangeo-integration-te-3eea/roles/monitoring.viewer/serviceAccount:pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/stackdriver.resourceMetadata.writer"]: Refreshing state... [id=pangeo-integration-te-3eea/roles/stackdriver.resourceMetadata.writer/serviceAccount:pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/artifactregistry.reader"]: Refreshing state... [id=pangeo-integration-te-3eea/roles/artifactregistry.reader/serviceAccount:pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/logging.logWriter"]: Refreshing state... [id=pangeo-integration-te-3eea/roles/logging.logWriter/serviceAccount:pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_project_iam_member.cluster_sa_roles["roles/monitoring.metricWriter"]: Refreshing state... [id=pangeo-integration-te-3eea/roles/monitoring.metricWriter/serviceAccount:pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_filestore_instance.homedirs[0]: Refreshing state... [id=projects/pangeo-integration-te-3eea/locations/us-central1-b/instances/pangeo-hubs-homedirs]�[0m
�[0m�[1mgoogle_compute_router.router[0]: Refreshing state... [id=projects/pangeo-integration-te-3eea/regions/us-central1/routers/pangeo-hubs-router]�[0m
�[0m�[1mgoogle_compute_firewall.iap_ssh_ingress[0]: Refreshing state... [id=projects/pangeo-integration-te-3eea/global/firewalls/allow-ssh]�[0m
�[0m�[1mgoogle_project_iam_member.identify_project_binding: Refreshing state... [id=pangeo-integration-te-3eea/projects/pangeo-integration-te-3eea/roles/pangeo_hubs_user_sa_role/serviceAccount:pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com]�[0m
�[0m�[1mgoogle_container_cluster.cluster: Refreshing state... [id=projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster]�[0m
�[0m�[1mgoogle_compute_router_nat.nat[0]: Refreshing state... [id=pangeo-integration-te-3eea/us-central1/pangeo-hubs-router/pangeo-hubs-router-nat]�[0m
�[0m�[1mgoogle_container_node_pool.dask_worker["large"]: Refreshing state... [id=projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/dask-large]�[0m
�[0m�[1mgoogle_container_node_pool.notebook["small"]: Refreshing state... [id=projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/nb-small]�[0m
�[0m�[1mgoogle_container_node_pool.dask_worker["small"]: Refreshing state... [id=projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/dask-small]�[0m
�[0m�[1mgoogle_container_node_pool.notebook["medium"]: Refreshing state... [id=projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/nb-medium]�[0m
�[0m�[1mgoogle_container_node_pool.notebook["large"]: Refreshing state... [id=projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/nb-large]�[0m
�[0m�[1mgoogle_container_node_pool.dask_worker["medium"]: Refreshing state... [id=projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/dask-medium]�[0m
�[0m�[1mgoogle_container_node_pool.core: Refreshing state... [id=projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/core-pool]�[0m
�[0m
�[1m�[36mNote:�[0m�[1m Objects have changed outside of Terraform�[0m
�[0mTerraform detected the following changes made outside of Terraform since the

last "terraform apply":
�[1m  # google_container_cluster.cluster�[0m has changed�[0m�[0m

�[0m  �[33m~�[0m�[0m resource "google_container_cluster" "cluster" {

�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster"

�[1m�[0mname�[0m�[0m                        = "pangeo-hubs-cluster"

�[90m# (27 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_pool {
        �[1m�[0mname�[0m�[0m                        = "core-pool"
      �[33m~�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 3 �[33m->�[0m �[0m2
        �[90m# (6 unchanged attributes hidden)�[0m�[0m





        �[90m# (5 unchanged blocks hidden)�[0m�[0m
    }
  �[33m~�[0m �[0mnode_pool {
        �[1m�[0mname�[0m�[0m                        = "nb-medium"
      �[33m~�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m1
        �[90m# (6 unchanged attributes hidden)�[0m�[0m





        �[90m# (5 unchanged blocks hidden)�[0m�[0m
    }
  �[33m~�[0m �[0mnode_pool {
        �[1m�[0mname�[0m�[0m                        = "nb-large"
      �[33m~�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 2 �[33m->�[0m �[0m1
        �[90m# (6 unchanged attributes hidden)�[0m�[0m





        �[90m# (5 unchanged blocks hidden)�[0m�[0m
    }
  �[33m~�[0m �[0mnode_pool {
        �[1m�[0mname�[0m�[0m                        = "dask-medium"
      �[33m~�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 14 �[33m->�[0m �[0m0
        �[90m# (6 unchanged attributes hidden)�[0m�[0m





        �[90m# (5 unchanged blocks hidden)�[0m�[0m
    }





    �[90m# (19 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.core�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_container_node_pool" "core" {

�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/core-pool"

�[1m�[0mname�[0m�[0m                        = "core-pool"

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 3 �[33m->�[0m �[0m2

�[90m# (9 unchanged attributes hidden)�[0m�[0m
    �[90m# (4 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.dask_worker["medium"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_container_node_pool" "dask_worker" {

�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/dask-medium"

�[1m�[0mname�[0m�[0m                        = "dask-medium"

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 14 �[33m->�[0m �[0m0

�[90m# (9 unchanged attributes hidden)�[0m�[0m
    �[90m# (4 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["large"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_container_node_pool" "notebook" {

�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/nb-large"

�[1m�[0mname�[0m�[0m                        = "nb-large"

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 2 �[33m->�[0m �[0m1

�[90m# (9 unchanged attributes hidden)�[0m�[0m
    �[90m# (4 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["medium"]�[0m has changed�[0m�[0m

�[0m  �[33m�[0m�[0m resource "google_container_node_pool" "notebook" {

�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/nb-medium"

�[1m�[0mname�[0m�[0m                        = "nb-medium"

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m1

�[90m# (9 unchanged attributes hidden)�[0m�[0m
    �[90m# (4 unchanged blocks hidden)�[0m�[0m
}

Unless you have made equivalent changes to your configuration, or ignored the

relevant attributes using ignore_changes, the following plan may include

actions to undo or respond to these changes.

�[90m

─────────────────────────────────────────────────────────────────────────────�[0m
Terraform used the selected providers to generate the following execution

plan. Resource actions are indicated with the following symbols:

�[31m-�[0m/�[32m+�[0m destroy and then create replacement

�[0m

Terraform will perform the following actions:
�[1m  # google_container_node_pool.dask_worker["large"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "dask_worker" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/dask-large" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroupManagers/gke-pangeo-hubs-cluster-dask-large-8741828e-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroups/gke-pangeo-hubs-cluster-dask-large-8741828e-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = 110 �[33m->�[0m �[0m(known after apply)

�[1m�[0mname�[0m�[0m                        = "dask-large"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m~�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.20.8-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-ssd" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

        �[90m# (1 unchanged block hidden)�[0m�[0m
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.dask_worker["medium"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "dask_worker" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/dask-medium" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroupManagers/gke-pangeo-hubs-cluster-dask-medium-f0702139-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroups/gke-pangeo-hubs-cluster-dask-medium-f0702139-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = 110 �[33m->�[0m �[0m(known after apply)

�[1m�[0mname�[0m�[0m                        = "dask-medium"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m~�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.20.8-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-ssd" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

        �[90m# (1 unchanged block hidden)�[0m�[0m
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.dask_worker["small"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "dask_worker" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/dask-small" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroupManagers/gke-pangeo-hubs-cluster-dask-small-abfb145b-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroups/gke-pangeo-hubs-cluster-dask-small-abfb145b-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = 110 �[33m->�[0m �[0m(known after apply)

�[1m�[0mname�[0m�[0m                        = "dask-small"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m~�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.20.8-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-ssd" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

        �[90m# (1 unchanged block hidden)�[0m�[0m
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["large"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "notebook" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/nb-large" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroupManagers/gke-pangeo-hubs-cluster-nb-large-7ec2a078-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroups/gke-pangeo-hubs-cluster-nb-large-7ec2a078-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = 110 �[33m->�[0m �[0m(known after apply)

�[1m�[0mname�[0m�[0m                        = "nb-large"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 1 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m~�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.20.8-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-standard" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

        �[90m# (1 unchanged block hidden)�[0m�[0m
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["medium"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "notebook" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/nb-medium" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroupManagers/gke-pangeo-hubs-cluster-nb-medium-04a21a74-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroups/gke-pangeo-hubs-cluster-nb-medium-04a21a74-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = 110 �[33m->�[0m �[0m(known after apply)

�[1m�[0mname�[0m�[0m                        = "nb-medium"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 1 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m~�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.20.8-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-standard" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

        �[90m# (1 unchanged block hidden)�[0m�[0m
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[1m  # google_container_node_pool.notebook["small"]�[0m must be �[1m�[31mreplaced�[0m�[0m

�[0m�[31m-�[0m/�[32m+�[0m�[0m resource "google_container_node_pool" "notebook" {

�[33m�[0m �[0m�[1m�[0mid�[0m�[0m                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/nb-small" �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0minstance_group_urls�[0m�[0m         = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroupManagers/gke-pangeo-hubs-cluster-nb-small-ef9b4ca7-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmanaged_instance_group_urls�[0m�[0m = [

�[31m-�[0m �[0m"https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroups/gke-pangeo-hubs-cluster-nb-small-ef9b4ca7-grp",

] �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mmax_pods_per_node�[0m�[0m           = 110 �[33m->�[0m �[0m(known after apply)

�[1m�[0mname�[0m�[0m                        = "nb-small"

�[32m+�[0m �[0m�[1m�[0mname_prefix�[0m�[0m                 = (known after apply)

�[33m�[0m �[0m�[1m�[0mnode_count�[0m�[0m                  = 0 �[33m->�[0m �[0m(known after apply)

�[33m�[0m �[0m�[1m�[0mnode_locations�[0m�[0m              = [

�[31m-�[0m �[0m"us-central1-b",

] �[33m->�[0m �[0m(known after apply)

�[32m+�[0m �[0m�[1m�[0moperation�[0m�[0m                   = (known after apply)

�[33m~�[0m �[0m�[1m�[0mversion�[0m�[0m                     = "1.20.8-gke.2101" �[33m->�[0m �[0m(known after apply)

�[90m# (4 unchanged attributes hidden)�[0m�[0m
  �[33m~�[0m �[0mnode_config {
      �[33m~�[0m �[0m�[1m�[0mdisk_size_gb�[0m�[0m      = 100 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mdisk_type�[0m�[0m         = "pd-standard" �[33m->�[0m �[0m"pd-balanced" �[31m# forces replacement�[0m�[0m
      �[33m~�[0m �[0m�[1m�[0mguest_accelerator�[0m�[0m = [] �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mimage_type�[0m�[0m        = "COS_CONTAINERD" �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mlocal_ssd_count�[0m�[0m   = 0 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmetadata�[0m�[0m          = {
          �[31m-�[0m �[0m"disable-legacy-endpoints" = "true"
        } �[33m->�[0m �[0m(known after apply)
        �[1m�[0mtags�[0m�[0m              = []
        �[90m# (6 unchanged attributes hidden)�[0m�[0m

      �[33m~�[0m �[0mshielded_instance_config {
          �[33m~�[0m �[0m�[1m�[0menable_integrity_monitoring�[0m�[0m = true �[33m->�[0m �[0m(known after apply)
          �[33m~�[0m �[0m�[1m�[0menable_secure_boot�[0m�[0m          = false �[33m->�[0m �[0m(known after apply)
        }

        �[90m# (1 unchanged block hidden)�[0m�[0m
    }

  �[33m~�[0m �[0mupgrade_settings {
      �[33m~�[0m �[0m�[1m�[0mmax_surge�[0m�[0m       = 1 �[33m->�[0m �[0m(known after apply)
      �[33m~�[0m �[0m�[1m�[0mmax_unavailable�[0m�[0m = 0 �[33m->�[0m �[0m(known after apply)
    }
    �[90m# (2 unchanged blocks hidden)�[0m�[0m
}

�[0m�[1mPlan:�[0m 6 to add, 0 to change, 6 to destroy.

�[0m�[33m╷�[0m�[0m

�[33m│�[0m �[0m�[1m�[33mWarning: �[0m�[0m�[1mArgument is deprecated�[0m

�[33m│�[0m �[0m

�[33m│�[0m �[0m�[0m  with google_filestore_instance.homedirs,

�[33m│�[0m �[0m  on storage.tf line 4, in resource "google_filestore_instance" "homedirs":

�[33m│�[0m �[0m   4:   zone = �[4mvar.zone�[0m�[0m

�[33m│�[0m �[0m

�[33m│�[0m �[0mDeprecated in favor of location.

�[33m│�[0m �[0m

�[33m│�[0m �[0m(and one more similar warning elsewhere)

�[33m╵�[0m�[0m

�[90m

─────────────────────────────────────────────────────────────────────────────�[0m
Note: You didn't use the -out option to save this plan, so Terraform can't

guarantee to take exactly these actions if you run "terraform apply" now.

[yuvipanda]

This is a destructive-ish change, so I'll have to apply this carefully and well timed.

[choldgraf]

cc @mmcky - we're playing around with speeding up the node creation a bit!

[sgibson91]

This LGTM - good luck with the apply!

[GeorgianaElena]

🤞🏼

[choldgraf]

This looks good to me - I'll defer to you all on the technical strategy here.

My one question is whether we can document the rationale for these decisions somewhere. For example, why pd-balanced for some and pd-ssd for others? Normally I would just say this can be a one-line comment above the config, but since this is spread out across many clusters I feel like this would be cumbersome to change.

Can anybody think of a quick way to document this? If not, I don't wanna block the PR on this, I just worry that we have lots of "implicit strategy" encoded in some of these infrastructure decisions that might be hard to reason with in the future without the context

[yuvipanda]

@choldgraf i had forgotten to push a commit - I have just made the decision for everything to just use pd-balanced, and documented it just above where that decision is made.

[yuvipanda]

I've taken advantage of a lull in usage to deploy this to the pilot-hubs and meom-ige cluster.

[yuvipanda]

Need to deploy this to cloudbank and pangeo-hubs still. Both have users right now.

[choldgraf]

This looks great to me, thanks Yuvi! And @mmcky it sounds like these changes have already been deployed to the cluster that you're using, so maybe keep an eye on any reports people make about slower starts and see if we are a bit faster than the 6-8 minute waits you were reporting before!

[choldgraf]

just wanna confirm that you're intentionally making these pd-ssd, since you mentioned in another comment that you had missed some of them

[yuvipanda]

Good catch, @choldgraf! Fixed :)

[mmcky]

thanks @choldgraf and @yuvipanda -- will report back from the user base. Really appreciate this.

[yuvipanda]

@mmcky thanks for helping us make the service better for everyone!

[yuvipanda]

This isn't deployed on pangeo-hubs and cloudbank, but am going to merge this now to make further PR work that touches this easier.

[mmcky]

@yuvipanda not sure if this is related to this merge. But for the Friday @ 3pm (AEST) Tutorial Lab for the ANU course the whole class went down and weren't able to use the anu jupyterhub. All the other tutorials have been fine. I am wondering if this time slot may have coincided with an upgrade (overnight US time). Just thought I'd feed that back.

[yuvipanda]

@mmcky oh no, I'm so sorry you ran into issues! I just investigated the hub and it is not related to this PR - #1135 has more information. It looks like you basically ran into #1103. This has happened twice now this week, so I'll bump up the priority in getting the fix deployed. Sorry for the inconvenience, and I'll let you know once we deploy the fix!

[mmcky]

Thanks so much @yuvipanda love your work.

[damianavila]

This isn't deployed on pangeo-hubs and cloudbank, but am going to merge this now to make further PR work that touches this easier.

@yuvipanda, is this follow-up task living in a new issue?

[yuvipanda]

@damianavila just opened up #1153