fix: added new tools such as oras and awscli, and script to check for…

… and delete packer cache (#24) * fix: added script to check for and delete packer cache * fix: added new tools such as oras and awscli, updated Terraform files * Apply suggestions from code review Co-authored-by: Karl Cardenas <[email protected]> * Update edge/vmware/clone_vm_template/delete-packer-cache.sh change error output destination * fix: added new tools such as oras and awscli, updated Terraform files… (#25) * fix: added new tools such as oras and awscli, updated Terraform files to include OCI registries * chore: updated the template and documentation * docs: minor update to the terraform.tfvars file * docs: minor update to the terraform.tfvars file --------- Co-authored-by: Karl Cardenas <[email protected]> --------- Co-authored-by: Lenny Chen <[email protected]> Co-authored-by: Karl Cardenas <[email protected]> Co-authored-by: caroldelwing <[email protected]>
spectrocloud · Nov 14, 2023 · d525fc6 · d525fc6
1 parent d36527f
commit d525fc6
Show file tree

Hide file tree

Showing 11 changed files with 177 additions and 102 deletions.
diff --git a/Dockerfile b/Dockerfile
@@ -18,6 +18,7 @@ ADD  static/defaults/registry-config.yml etc/spectro/config.yml
 ARG PALETTE_CLI_VERSION
 ARG PALETTE_EDGE_VERSION
 ARG PACKER_VERSION=1.9.4
+ARG ORAS_VERSION=1.0.0
 
 ENV REGISTRY_LOG_LEVEL=info
 ENV REGISTRY_AUTH=htpasswd
@@ -29,7 +30,7 @@ COPY --from=server /etc/spectro/config.yml /etc/spectro/config.yml
 
 RUN adduser -H -u 1002 -D appuser appuser && \
     apk update && \
-    apk add --no-cache bash curl git terraform openssl jq bind-tools wget ca-certificates nano
+    apk add --no-cache bash curl git terraform openssl jq bind-tools wget ca-certificates nano aws-cli xorriso govc
 
 RUN  wget https://software.spectrocloud.com/spectro-registry/cli/v$PALETTE_CLI_VERSION/linux/spectro && \
         mv spectro /usr/local/bin/spectro && \
@@ -50,13 +51,17 @@ RUN  wget https://software.spectrocloud.com/spectro-registry/cli/v$PALETTE_CLI_V
         wget https://software.spectrocloud.com/stylus/v$PALETTE_EDGE_VERSION/cli/linux/palette-edge && \
         mv palette-edge /usr/local/bin/palette-edge && \
         chmod +x /usr/local/bin/palette-edge && \
+        wget https://github.com/oras-project/oras/releases/download/v${ORAS_VERSION}/oras_${ORAS_VERSION}_linux_amd64.tar.gz && \
+        mkdir -p oras-install/ && \
+        tar -zxf oras_${ORAS_VERSION}_*.tar.gz -C oras-install/ && \
+        mv oras-install/oras /usr/local/bin/ && \
+        rm -rf oras_${ORAS_VERSION}_*.tar.gz oras-install/ && \
         git clone https://github.com/spectrocloud/CanvOS.git && \
         rm -rf /var/cache/apk/* 
 
 ADD https://releases.hashicorp.com/packer/${PACKER_VERSION}/packer_${PACKER_VERSION}_linux_amd64.zip /usr/local/sbin/
 RUN unzip /usr/local/sbin/packer_${PACKER_VERSION}_linux_amd64.zip -d /usr/local/sbin && \
     rm -rf /usr/local/sbin/packer_${PACKER_VERSION}_linux_amd64.zip
-RUN apk add xorriso govc
 EXPOSE 5000
 
 CMD ["/bin/bash"]
diff --git a/docs/docker.md b/docs/docker.md
@@ -40,6 +40,8 @@ The Docker image includes the following tools.
 | `nano`        | A text editior for Unix-like computing systems or operating environments using a command line interface|
 | `packer`      | Hashicorp Image Builder for building Edge Native images        |
 | `GOVC`        | Tool for interracting with VMware Vsphere via API              |
+| `aws-cli`     | AWS command-line interface                                     |
+| `oras`        | Tool for pushing and pulling OCI artifacts to and from OCI registries|
 
 ### Spectro Cloud Pack Registry Server
 
@@ -63,5 +65,5 @@ If you are wanting to build the docker image locally you must provide the the bu
 
 
 ```shell
-docker build --build-arg PALETTE_VERSION=4.0.1 --build-arg PALETTE_CLI_VERSION=4.0.1  --build-arg PALETTE_EDGE_VERSION=3.4.3 --build-arg PACKER_VERSION=1.9.4 -t tutorials .
+docker build --build-arg PALETTE_VERSION=4.0.2 --build-arg PALETTE_CLI_VERSION=4.0.1  --build-arg PALETTE_EDGE_VERSION=3.4.3 --build-arg PACKER_VERSION=1.9.4 -t tutorials .
 ```
diff --git a/edge/vmware/clone_vm_template/delete-packer-cache.sh b/edge/vmware/clone_vm_template/delete-packer-cache.sh
@@ -0,0 +1,22 @@
+#!/bin/bash
+
+source /edge/vmware/clone_vm_template/setenv.sh
+
+# Check if the packer cache exists
+CACHE_EXISTS=$(govc datastore.ls -ds=$vcenter_datastore /packer_cache 2>&1)
+
+if [ -z "$CACHE_EXISTS" ]; then
+  echo "Existing packer cache not found. Nothing to delete."
+else
+  # Cache exists, attempt to delete it
+  govc datastore.rm -ds=$vcenter_datastore /packer_cache
+  RESULT=$?
+  if [ $RESULT -eq 0 ]; then
+    echo "Deleted previous packer cache."
+    exit 0
+  else
+    echo "Failed to delete packer cache."
+    exit 1
+  fi
+fi
+
diff --git a/terraform/pack-tf/README.md b/terraform/pack-tf/README.md
@@ -3,30 +3,33 @@
 This folder contains the demo code for the [Create and Deploy a Custom Add-On Pack](https://docs.spectrocloud.com/registries-and-packs/create-pack) tutorial.
 Use the code in this directory with the tutorial. 
 
+
+To get started, review the [prerequisites](#prerequisites). Next, follow the steps in the tutorial to deploy the cluster and custom add-on pack.
+ Use the [terraform.tfvars](./terraform.tfvars) file to provide the required variables. Once you populate the variables, you can deploy the cluster and custom add-on pack.
+
 ## Prerequisites
 You will need the following things before getting started:
-1. Spectro Cloud API key generated from the Palette.
+1. A Palette API key.
 2. A cloud account added to your Palette project settings. 
-3. An SSH key created in the region where you will deploy the cluster.
-
-Note that the Terraform code, when executed, will deploy the resources to the **AWS** cloud service provider. 
-To deploy your resource to Azure or Google Cloud, use the specific layer details outlined in the **Cloud Service Provider Configurations** section below.
+3. An AWS Key pair or SSH key created in the region where you will deploy the cluster.
 
+> [!NOTE]
+> This Terraform code will deploy the resources specified in the template to **AWS**.
 
-## Cloud Service Provider Configurations
-The code uses the data resources, the core infrastructure layers, defined in the **data.tf** to deploy the `spectrocloud_cluster_profile.profile` resource to the AWS. Here are the pack details for each infrastructure layer. 
 
-### AWS Config
+## Requirements
 
 | Name | Version |
 |------|---------|
-| <a name="requirement_spectrocloud"></a> [spectrocloud](#requirement\_spectrocloud) | >= 0.11.1 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 5.0 |
+| <a name="requirement_spectrocloud"></a> [spectrocloud](#requirement\_spectrocloud) | >= 0.16.1 |
 
 ## Providers
 
 | Name | Version |
 |------|---------|
-| <a name="provider_spectrocloud"></a> [spectrocloud](#provider\_spectrocloud) | >= 0.11.1 |
+| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.25.0 |
+| <a name="provider_spectrocloud"></a> [spectrocloud](#provider\_spectrocloud) | 0.16.1 |
 
 ## Modules
 
@@ -38,6 +41,7 @@ No modules.
 |------|------|
 | [spectrocloud_cluster_aws.cluster](https://registry.terraform.io/providers/spectrocloud/spectrocloud/latest/docs/resources/cluster_aws) | resource |
 | [spectrocloud_cluster_profile.profile](https://registry.terraform.io/providers/spectrocloud/spectrocloud/latest/docs/resources/cluster_profile) | resource |
+| [aws_availability_zones.available](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones) | data source |
 | [spectrocloud_cloudaccount_aws.account](https://registry.terraform.io/providers/spectrocloud/spectrocloud/latest/docs/data-sources/cloudaccount_aws) | data source |
 | [spectrocloud_pack.cni](https://registry.terraform.io/providers/spectrocloud/spectrocloud/latest/docs/data-sources/pack) | data source |
 | [spectrocloud_pack.csi](https://registry.terraform.io/providers/spectrocloud/spectrocloud/latest/docs/data-sources/pack) | data source |
@@ -47,21 +51,25 @@ No modules.
 | [spectrocloud_pack.ubuntu](https://registry.terraform.io/providers/spectrocloud/spectrocloud/latest/docs/data-sources/pack) | data source |
 | [spectrocloud_registry.hellouniverseregistry](https://registry.terraform.io/providers/spectrocloud/spectrocloud/latest/docs/data-sources/registry) | data source |
 | [spectrocloud_registry.public_registry](https://registry.terraform.io/providers/spectrocloud/spectrocloud/latest/docs/data-sources/registry) | data source |
+| [spectrocloud_registry_oci.hellouniverseregistry](https://registry.terraform.io/providers/spectrocloud/spectrocloud/latest/docs/data-sources/registry_oci) | data source |
 
 ## Inputs
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_aws_region_name"></a> [aws\_region\_name](#input\_aws\_region\_name) | Choose the AWS region. | `string` | n/a | yes |
-| <a name="input_cluster_cloud_account_aws_name"></a> [cluster\_cloud\_account\_aws\_name](#input\_cluster\_cloud\_account\_aws\_name) | Choose the AWS account integrated with Spektro Palette. | `string` | n/a | yes |
-| <a name="input_cluster_name"></a> [cluster\_name](#input\_cluster\_name) | Give the cluster a name. | `string` | `"pack-tutorial-cluster"` | no |
-| <a name="input_cluster_profile_description"></a> [cluster\_profile\_description](#input\_cluster\_profile\_description) | Provide a description. | `string` | `"My cluster profile as part of the packs tutorial."` | no |
-| <a name="input_cluster_profile_name"></a> [cluster\_profile\_name](#input\_cluster\_profile\_name) | Give the cluster-profile a name. | `string` | `"pack-tutorial-profile"` | no |
+| <a name="input_aws_az_names"></a> [aws\_az\_names](#input\_aws\_az\_names) | Provide a list of AWS Availability Zones. For example: ['us-east-1a', 'us-east-1b', 'us-east-1c'] | `list(string)` | `[]` | no |
+| <a name="input_aws_region_name"></a> [aws\_region\_name](#input\_aws\_region\_name) | Specify the AWS region where you want to deploy the cluster. | `string` | n/a | yes |
+| <a name="input_cluster_cloud_account_aws_name"></a> [cluster\_cloud\_account\_aws\_name](#input\_cluster\_cloud\_account\_aws\_name) | Specify the AWS account integrated with Palette. Use the same name as the one used in the Palette project settings. | `string` | n/a | yes |
+| <a name="input_cluster_name"></a> [cluster\_name](#input\_cluster\_name) | The name of the cluster. | `string` | `"pack-tutorial-cluster"` | no |
+| <a name="input_cluster_profile_description"></a> [cluster\_profile\_description](#input\_cluster\_profile\_description) | Provide a description of the cluster profile. | `string` | `"My cluster profile as part of the packs tutorial."` | no |
+| <a name="input_cluster_profile_name"></a> [cluster\_profile\_name](#input\_cluster\_profile\_name) | The name of the cluster profile. | `string` | `"pack-tutorial-profile"` | no |
 | <a name="input_custom_addon_pack"></a> [custom\_addon\_pack](#input\_custom\_addon\_pack) | Custom add-on pack name. | `string` | `"hellouniverse"` | no |
 | <a name="input_custom_addon_pack_version"></a> [custom\_addon\_pack\_version](#input\_custom\_addon\_pack\_version) | Custom add-on pack version. | `string` | `"1.0.0"` | no |
-| <a name="input_private_pack_registry"></a> [private\_pack\_registry](#input\_private\_pack\_registry) | Private pack registry server name. | `string` | `"private-pack-registry"` | no |
-| <a name="input_ssh_key_name"></a> [ssh\_key\_name](#input\_ssh\_key\_name) | Choose the AWS region. | `string` | n/a | yes |
+| <a name="input_instance_type"></a> [instance\_type](#input\_instance\_type) | Specify the AWS instance type. | `string` | `"m4.xlarge"` | no |
+| <a name="input_private_pack_registry"></a> [private\_pack\_registry](#input\_private\_pack\_registry) | The name of the private pack registry server. | `string` | n/a | yes |
+| <a name="input_ssh_key_name"></a> [ssh\_key\_name](#input\_ssh\_key\_name) | Specify the AWS Keypair available in the AWS region where you want to deploy the cluster. | `string` | n/a | yes |
 | <a name="input_tags"></a> [tags](#input\_tags) | The default tags to apply to Palette resources | `list(string)` | <pre>[<br>  "spectro-cloud-education",<br>  "app:hello-universe",<br>  "terraform_managed:true"<br>]</pre> | no |
+| <a name="input_use_oci_registry"></a> [use\_oci\_registry](#input\_use\_oci\_registry) | Set the use of OCI registry to true or false. If you are not using an OCI registry, set this value to false. | `bool` | `true` | no |
 
 ## Outputs
 

diff --git a/terraform/pack-tf/cluster.tf b/terraform/pack-tf/cluster.tf
@@ -1,14 +1,14 @@
 resource "spectrocloud_cluster_aws" "cluster" {
-  name              = var.cluster_name
-  tags              = var.tags
-  cloud_account_id  = data.spectrocloud_cloudaccount_aws.account.id
-  depends_on        = [spectrocloud_cluster_profile.profile]
+  name             = var.cluster_name
+  tags             = var.tags
+  cloud_account_id = data.spectrocloud_cloudaccount_aws.account.id
+  depends_on       = [spectrocloud_cluster_profile.profile]
 
   cloud_config {
-    ssh_key_name = var.ssh_key_name         
+    ssh_key_name = var.ssh_key_name
     region       = var.aws_region_name
   }
-  
+
   cluster_profile {
     id = spectrocloud_cluster_profile.profile.id
   }
@@ -32,9 +32,9 @@ resource "spectrocloud_cluster_aws" "cluster" {
     control_plane_as_worker = true
     name                    = "master-pool"
     count                   = 1
-    instance_type           = "m4.large"
+    instance_type           = var.instance_type
     disk_size_gb            = 60
-    azs                     = ["us-east-2a"]     
+    azs                     = local.azs
   }
 
   ##############################
@@ -48,8 +48,8 @@ resource "spectrocloud_cluster_aws" "cluster" {
     }
     name          = "worker-basic"
     count         = 1
-    instance_type = "m5.large"
-    azs           = ["us-east-2a"]
+    instance_type = var.instance_type
+    azs           = local.azs
   }
 
 }
diff --git a/terraform/pack-tf/data.tf b/terraform/pack-tf/data.tf
@@ -2,7 +2,7 @@
 # Data resources for the profile
 ####################################
 data "spectrocloud_registry" "public_registry" {
-  name      = "Public Repo"
+  name = "Public Repo"
 }
 
 ####################################
@@ -12,51 +12,66 @@ data "spectrocloud_registry" "public_registry" {
 # Refer to the "Cloud Service Provider Configurations" section in the README for more details.  
 ####################################
 data "spectrocloud_pack" "ubuntu" {
-  name      = "ubuntu-aws"
-  version   = "20.04"
+  name         = "ubuntu-aws"
+  version      = "22.04"
   registry_uid = data.spectrocloud_registry.public_registry.id
 }
 
 data "spectrocloud_pack" "k8s" {
-  name      = "kubernetes"
-  version   = "1.24.10"
+  name         = "kubernetes"
+  version      = "1.28.2"
   registry_uid = data.spectrocloud_registry.public_registry.id
 }
 
 data "spectrocloud_pack" "cni" {
-  name      = "cni-calico"
-  version   = "3.25.0"
+  name         = "cni-calico"
+  version      = "3.26.1"
   registry_uid = data.spectrocloud_registry.public_registry.id
 }
 
 data "spectrocloud_pack" "csi" {
-  name      = "csi-aws-ebs"
-  version   = "1.16.0"
+  name         = "csi-aws-ebs"
+  version      = "1.22.0"
   registry_uid = data.spectrocloud_registry.public_registry.id
 }
 
 ####################################
 # Add-On Layers
 ####################################
 data "spectrocloud_pack" "spectro-proxy" {
-  name      = "spectro-proxy"
-  version   = "1.3.0"
-  type      = "spectro"
+  name         = "spectro-proxy"
+  version      = "1.4.1"
+  type         = "spectro"
+  registry_uid = data.spectrocloud_registry.public_registry.id
 }
 
+# Select the correct registry (OCI or non-OCI)
+
 data "spectrocloud_pack" "hellouniverse" {
-  name      = var.custom_addon_pack
-  version   = var.custom_addon_pack_version
-  registry_uid = data.spectrocloud_registry.hellouniverseregistry.id
+  name         = var.custom_addon_pack
+  version      = var.custom_addon_pack_version
+  registry_uid = var.use_oci_registry ? data.spectrocloud_registry_oci.hellouniverseregistry[0].id : data.spectrocloud_registry.hellouniverseregistry[0].id
 }
 
 data "spectrocloud_registry" "hellouniverseregistry" {
+  count = var.use_oci_registry ? 0 : 1
+  name  = var.private_pack_registry
+}
+
+data "spectrocloud_registry_oci" "hellouniverseregistry" {
+  count = var.use_oci_registry ? 1 : 0
   name  = var.private_pack_registry
 }
 
 ####################################
 # Data resources for the cluster
 ####################################
 data "spectrocloud_cloudaccount_aws" "account" {
-  name  = var.cluster_cloud_account_aws_name
-}
+  name = var.cluster_cloud_account_aws_name
+}
+
+
+####################################
+# AWS 
+####################################
+data "aws_availability_zones" "available" {}