Enable support for Azure US Government for provider-azure

[aabond]

How to categorize this issue?

/area control-plane
/area usability
/area robustness
/kind enhancement
/priority 1
/platform azure

What would you like to be added:

Need the ability to deploy shoot clusters on Azure US Government Cloud. It seems the Azure infrastructure secret needs the ability to distinguish between Azure commercial and Azure US Government clouds. This parameter should then be provided to the underlying terraform code to switch to the Azure US Government API endpoint.

Why is this needed:

SAP NS2 needs to deploy SAP FieldGlass on to Azure US Government Cloud.

Current error being displayed while trying to deploy shoot cluster to Azure US Government:

Flow "Shoot cluster reconciliation" encountered task errors: [task "Waiting until shoot infrastructure has been reconciled" failed: Error while waiting for Infrastructure shoot--fglns2can--test-cluster/test-cluster to become ready: error during reconciliation: Error reconciling infrastructure: failed to apply the terraform config: Terraform execution for command 'apply' could not be completed:

* Error building account: Error getting authenticated object ID: Error listing Service Principals: autorest.DetailedError{Original:adal.tokenRefreshError{message:"adal: Refresh request failed. Status Code = '400'. Response body: {\"error\":\"invalid_request\",\"error_description\":\"AADSTS900382: Confidential Client is not supported in Cross Cloud request.\\r\\nTrace ID: <omitted>\\r\\nCorrelation ID: <omitted>\\r\\nTimestamp: 2021-10-06 18:48:08Z\",\"error_codes\":[900382],\"timestamp\":\"2021-10-06 18:48:08Z\",\"trace_id\":\"<omitted>\",\"correlation_id\":\"<omitted>\"} Endpoint https://login.microsoftonline.com/<omitted>/oauth2/token?api-version=1.0", resp:(*http.Response)(0xc0001377a0)}, PackageType:"azure.BearerAuthorizer", Method:"WithAuthorization", StatusCode:400, Message:"Failed to refresh the Token for request to https://graph.windows.net/<omitted>/servicePrincipals?%24filter=appId+eq+%27<omitted>%27&api-version=1.6", ServiceError:[]uint8(nil), Response:(*http.Response)(0xc0001377a0)}
  on tf/main.tf line 1, in provider "azurerm":
   1: provider "azurerm" {] Operation will be retried.

Further notes:
I came across this post when searching for the error:

https://serverfault.com/questions/1064253/azure-runbook-fails-to-connect-confidential-client-is-not-supported-in-cross

Which basically said adding the flag -Environment AzureUSGovernment solved their issue.

[dkistner]

Hello @aabond,

thanks for the request.
I guess there will be adaptions on the configuration for multiple components are required.
First of course the obvious things to do:

For the cloud-controller-manager we will need to update the cloud-provider-config by adding the cloud="USGovernmentCloud". Same for the csi/kube-controller-manager cloud-provider-configs.
See here: https://kubernetes-sigs.github.io/cloud-provider-azure/install/configs

We need also to adapt the mcm provider for Azure so that the correct environment is chosen: https://github.com/gardener/machine-controller-manager-provider-azure/blob/master/pkg/spi/azure.go#L36

And last we need also adapt the internal Azure client setup of the Azure extension.

[dkistner]

/assign

[dkistner]

/unassign as currently there is no active work on this