Merge pull request #155 from Azure/dcint-avm

Dcint avm

.vscode/cspell.json

@@ -107,7 +107,7 @@
         "xattr",
         "smartbrain"
     ],
-    "ignoreWords": ["ACCOUNTNAME","OIDCISSUERURL","UNIQUESTRING", "outfile"],
+    "ignoreWords": ["ACCOUNTNAME","OIDCISSUERURL","UNIQUESTRING", "outfile","contoso"],
     "import": [],
     "enableFiletypes": [
         "!css",

Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/02-EID/ad_groups.ps1

@@ -0,0 +1,20 @@
+param(
+    [Parameter(Mandatory=$true)]
+    [string]$appdevs,
+    [Parameter(Mandatory=$true)]
+    [string]$aksops
+)
+
+# checking if Azure module is installed
+$isInstalled = $false
+if(Get-InstalledModule -Name Az.Resources -ErrorAction SilentlyContinue){
+    $isInstalled = $true
+}
+
+if($isInstalled){
+    New-AzADGroup -DisplayName $appdevs -MailNickname $appdevs
+    New-AzADGroup -DisplayName $aksops -MailNickname $aksops
+}
+else {
+    Write-Output "Azure PowerShell not installed. Installation steps in: https://learn.microsoft.com/powershell/azure/install-az-ps"
+}

Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/02-eid.md

@@ -0,0 +1,81 @@
+# Prerequisites and Microsoft Entra ID 
+
+This is the starting point for the instructions on deploying the [AKS Baseline private cluster reference implementation](../README.md). There is required access and tooling you'll need in order to accomplish this. Follow the instructions below and on the subsequent pages so that you can get your environment ready to proceed with the AKS cluster creation.
+
+## Steps
+
+1. Latest [Azure CLI installed](https://learn.microsoft.com/cli/azure/install-azure-cli?view=azure-cli-latest) (must be at least 2.59), or you can perform this from Azure Cloud Shell by clicking below.
+1. An Azure subscription.
+
+   The subscription used in this deployment cannot be a [free account](https://azure.microsoft.com/free); it must be a standard EA, pay-as-you-go, or Visual Studio benefit subscription. This is because the resources deployed here are beyond the quotas of free subscriptions.
+
+   > :warning: The user or service principal initiating the deployment process _must_ have the following minimal set of Azure Role-Based Access Control (RBAC) roles:
+   >
+   > * [Contributor role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#contributor) is _required_ at the subscription level to have the ability to create resource groups and perform deployments.
+   > * [User Access Administrator role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator) is _required_ at the subscription level since you'll be performing role assignments to managed identities across various resource groups.
+
+1. **This step only applies if you are creating a new Microsoft Entra group for this deployment. If you have one already existing and you are a part of it, you can skip this prerequisite, and the remaining steps in this page, move on to the next page by clicking on the link at the bottom**.
+
+   A Microsoft Entra ID tenant to associate your Kubernetes RBAC Cluster API authentication to.
+
+   > :warning: The user or service principal initiating the deployment process _must_ have the following minimal set of Microsoft Entra ID permissions assigned:
+   >
+   > * Microsoft Entra [User Administrator](https://learn.microsoft.com/entra/identity/role-based-access-control/permissions-reference#user-administrator-permissions) is _required_ to create a "break glass" AKS admin Microsoft Entra security group and user. Alternatively, you could get your Microsoft Entra admin to create this for you when instructed to do so.
+   >   * If you are not part of the User Administrator group in the tenant associated to your Azure subscription, please consider [creating a new tenant](https://learn.microsoft.com/entra/fundamentals/create-new-tenant#create-a-new-tenant-for-your-organization) to use while evaluating this implementation. The Microsoft Entra tenant backing your cluster's API RBAC does NOT need to be the same tenant associated with your Azure subscription.
+
+# Create Microsoft Entra groups for AKS
+
+Before creating the Microsoft Entra ID integrated cluster, groups must be created that can be later mapped to the Built-In Roles of "Azure Kubernetes Service Cluster User Role" and "Azure Kubernetes Service RBAC Cluster Admin".
+
+Depending on the needs of your organization, you may have a choice of existing groups to use or a new groups may need to be created for each cluster deployment.  
+
+Navigate to "/AKS-Secure-Baseline-Private-AVM/Bicep/02-EID" folder
+
+```azurecli
+cd ./Scenarios/AKS-Secure-Baseline-Private-AVM/Bicep/02-EID
+```
+
+Use the Azure CLI or Azure PowerShell to create the Microsoft Entra groups. Replace the Microsoft Entra group names below with the name of the Microsoft Entra groups you want to create, such as AKS_ES_dev, AKS_ES_ops. There should be no space in the names.
+
+# [CLI](#tab/CLI)
+
+```azurecli
+appdevs=<Microsoft Entra group name>
+aksops=<Microsoft Entra group name>
+
+az ad group create --display-name $appdevs --mail-nickname $appdevs
+az ad group create --display-name $aksops --mail-nickname $aksops
+```
+
+# [PowerShell](#tab/PowerShell)
+
+Running the command to create the new Microsoft Entra groups requires the New-AzADGroup cmdlet. More details can be found [here](https://learn.microsoft.com/powershell/azure/install-az-ps).
+
+Install New-AzADGroup cmdlet
+
+```azurepowershell
+Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
+Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force
+```
+
+Run the command below to create two new Microsoft Entra groups in your tenant. 
+
+```azurepowershell
+./ad_groups.ps1 -appdevs <App Dev Group> -aksops <AKS Operations Team>
+```
+
+## Ensure you are part of the Microsoft Entra group you just created or pointed to
+
+1. Go to Azure portal and type Microsoft Entra ID
+2. Select **Microsoft Entra ID**
+3. Click on **Groups** in the left blade
+4. Select the Admin User group you just created. For the default name, this should be *AKS App Admin Team*
+5. Click on **Members** in the left blade
+6. ![Location of private link for keyvault](../media/adding-to-eid-group.png)
+7. Click **+ Add members**
+8. Enter your name in the search bar and select your user(s)
+9. Click **Select**
+
+### Next step
+
+:arrow_forward: [Creation of Hub Network & its respective Components](./03-network-hub.md)

Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/03-network-hub.md

@@ -0,0 +1,41 @@
+# Deploy this scenario using the AKS AVM
+
+This scenario will be deployed using Azure Verified Modules (AVM). AVM is an initiative to consolidate and set the standards for what a good Infrastructure-as-Code module looks like.
+
+Modules will then align to these standards, across languages (Bicep, Terraform etc.) and will then be classified as AVMs and available from their respective language specific registries. These AVMs are fully supported by Microsoft and customers can use them in their production Terraform Code. For more information about AVM, check out the [AVM website](https://azure.github.io/Azure-Verified-Modules/).
+
+# Create the Hub Network
+
+If you haven't yet, clone the repo and cd to the appropriate folder
+
+```bash
+git clone https://github.com/Azure/AKS-Landing-Zone-Accelerator
+cd ./Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/02-EID
+```
+
+The following will be created:
+
+* Resource Group for Hub Networking
+* Hub VNET
+* Azure Firewall
+* Azure Bastion Host
+
+Navigate to "/Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/" folder
+
+```bash
+cd ./03-Network-Hub
+```
+
+Review the "input.tf" file and update the variable values if required according to your needs. Pay attentions to VNET address prefixes and subnets so it doesn't overlap Spoke VNET in further steps. Also, please pay attention to update Subnet prefix for AKS cluster in Spoke VNET in the further steps to be planned and update in this file.
+
+Once the files are updated, deploy using terraform cli.
+
+# [CLI](#tab/CLI)
+
+```terracli
+terraform init
+terraform plan -out main.tfplan
+terraform apply main.tfplan -auto-approve
+```
+
+:arrow_forward: [Creation of Spoke Network & its respective Components](./04-network-lz.md)

Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/04-network-lz.md

@@ -0,0 +1,28 @@
+# Create the Landing Zone Network
+
+The following will be created:
+
+* Resource Group for Landing Zone Networking
+* Spoke Virtual Network and Subnets
+* Peering of Hub and Spoke Networks
+* Private DNS Zones
+* Application Gateway
+* NSGs for AKS subnet and Application Gateway subnet
+
+Navigate to "/Scenarios/AKS-Secure-Baseline-PrivateCluster-AVM/Terraform/" folder
+
+```bash
+cd ./04-Network-LZ
+```
+
+Review "input.tf" and update the variable values as required. Please note to verify the Azure Firewall Private IP from the previous deployment in step 03. Once the files are updated, deploy using terraform cli.
+
+# [CLI](#tab/CLI)
+
+```terracli
+terraform init
+terraform plan -out main.tfplan
+terraform apply main.tfplan -auto-approve
+```
+
+:arrow_forward: [Creation of Supporting Components for AKS](./05-aks-supporting.md)

Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/05-aks-supporting.md

@@ -0,0 +1,27 @@
+# Create resources that support AKS
+
+The following will be created:
+
+* Azure Container Registry
+* Azure Key Vault
+* Private Link Endpoints for ACR and Key Vault
+* Related DNS settings for private endpoints
+* A managed identity
+
+Navigate to "/Scenarios/AKS-Secure-Baseline-PrivateCluster/Terraform/" folder
+
+```bash
+cd ./05-AKS-supporting
+```
+
+Review "input.tf" and update the variable values as required. Once the files are updated, deploy using terraform cli.
+
+# [CLI](#tab/CLI)
+
+```terracli
+terraform init
+terraform plan -out main.tfplan
+terraform apply main.tfplan -auto-approve
+```
+
+:arrow_forward: [Creation of AKS & enabling Addons](./06-aks-cluster.md)

Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/06-AKS-Cluster/input.tf

@@ -24,9 +24,9 @@ variable "vnetHubName" {
   default = "vnet-hub"
 }
 
-variable "admin-group-object-ids" {
+variable "adminGroupObjectIds" {
   type    = string
-  default = "d1553d93-3b9f-4d52-a28b-e4a4a27c114c"
+  default = " "
 
 }
 

Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/06-AKS-Cluster/main.tf

@@ -105,7 +105,7 @@ resource "azurerm_kubernetes_cluster" "aks-cluster" {
   }
   azure_active_directory_role_based_access_control {
     managed                = true
-    admin_group_object_ids = [var.admin-group-object-ids]
+    admin_group_object_ids = [var.adminGroupObjectIds]
   }
 
   default_node_pool {

Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/06-aks-cluster.md

@@ -0,0 +1,82 @@
+# Create resources for the AKS Cluster
+
+The following will be created:
+
+* AKS Cluster with KeyVault, nginx and monitoring addons
+* Log Analytics Workspace
+* ACR Access to the AKS Cluster
+* Updates to KeyVault access policy with AKS keyvault addon
+
+Navigate to "/Scenarios/AKS-Secure-Baseline-Private-AVM/Terraform/" folder
+
+```bash
+cd ./06-AKS-cluster
+```
+
+To create an AKS cluster that can use the Secrets Store CSI Driver, you must enable the AKS-AzureKeyVaultSecretsProvider feature flag on your subscription. Register the AKS-AzureKeyVaultSecretsProvider feature flag by using the az feature register command, as shown below
+
+```bash
+az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AKS-AzureKeyVaultSecretsProvider')].{Name:name,State:properties.state}"
+```
+
+if not enter the command below to enable it
+
+```bash
+az feature register --namespace "Microsoft.ContainerService" --name "AKS-AzureKeyVaultSecretsProvider"
+```
+
+It takes a few minutes for the status to show *Registered*. Verify the registration status by using the [az feature list](https://learn.microsoft.com/cli/azure/feature#az_feature_list) command:
+
+```bash
+az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AKS-AzureKeyVaultSecretsProvider')].{Name:name,State:properties.state}"
+```
+
+When ready, refresh the registration of the *Microsoft.ContainerService* resource provider by using the [az provider register](https://learn.microsoft.com/cli/azure/provider#az_provider_register) command:
+
+```bash
+az provider register --namespace Microsoft.ContainerService
+```
+
+There are a few additional Azure Providers and features that needs to be registered as well. Follow the same steps above for the following providers and features:
+
+* Microsoft.ContainerService
+* AKS-AzureKeyVaultSecretsProvider
+* Microsoft.OperationsManagement
+* Microsoft.OperationalInsights
+* EncryptionAtHost
+
+Here is a list with all required providers or features to be registered:
+
+```bash
+az provider register --namespace Microsoft.ContainerService
+az provider register --namespace Microsoft.OperationsManagement
+az provider register --namespace Microsoft.OperationalInsights
+az feature register --namespace "Microsoft.ContainerService" --name "AKS-AzureKeyVaultSecretsProvider"
+az feature register --namespace Microsoft.Compute --name EncryptionAtHost
+```
+
+> :warning: Don't move ahead to the next steps until all providers are registered.
+
+There is a admin group you need to change in input.tf
+
+    - Admin group which will grant the role "Azure Kubernetes Service Cluster Admin Role". The variable name is: admin-group-object-ids. 
+
+## Deploy the cluster
+
+Review "**input.tf**" file and update the variable values as required. Please make sure to update the Microsoft Entra ID group IDs with ones created in Step 02 and kubernetesVersion in the variables file. Once the files are updated, deploy using terraform cli
+
+The Kubernetes community releases minor versions roughly every three months. AKS has it own supportability policy based in the community releases. Before proceeding with the deployment, check the latest version reviewing the [supportability doc](https://learn.microsoft.com/azure/aks/supported-kubernetes-versions). You can also check the latest version by using the following command:
+
+```azurecli
+az aks get-versions -l $REGION
+```
+
+# [CLI](#tab/CLI)
+
+```terracli
+terraform init
+terraform plan -out main.tfplan
+terraform apply main.tfplan -auto-approve
+```
+
+:arrow_forward: [Deploy a Basic Workload](./07-workload.md)