Add Azure support #1

takuro-sato · 2024-01-23T16:14:54Z

Adding Azure support

This PR is adding support for deployment of Google's Bidding and Auction Services on Azure.

Azure Privacy Sandbox architecture: https://1drv.ms/w/s!AmI-86sms1pYqJ5Uqgo5Qv2Ynmrcmw?e=BDC8BH
(We'll make a PR for the document in https://github.com/privacysandbox/protected-auction-services-docs/tree/main in future)

Now B&A services can fetch private and public HPKE keys from an Azure KMS, specifically designed to support the B&A services, and handle test requests.
To try this changes locally, please visit here.

The PR for bidding-auction-servers repository: privacysandbox/bidding-auction-servers#9

Changes

Add Azure support
- Add Azure configurations. e.g. --platform=azure (for Bazel), kAzure (C++ enum value).
- Add Azure implementation for selected interfaces under cpio/client_providers/
Add aci_attestation_lib library to fetch attestation in Azure Confidential ACI. We implemented the core functionality of:
- kms_client_provider
- parameter_client_provider
- private_key_fetcher_provider
- private_key_fetcher_provider.
On the other hand, we haven't started implementing the following interfaces:
- auth_token_provider
- role_credentials_provider
Also we return dummy values for instance_client_provider for now. Please see "TODOs for future PRs" section for the details.

TODOs for future PRs

auth_token_provider is not implemented yet. So private_key_fetcher_provider and kms_client_provider are not using authentication token when accessing Azure KMS. It will be implemented using Azure Active Directory (Azure AD).
instance_client_provider currently returns dummy values. It will be implemented properly after Azure auth_token_provider is ready.
URL to decrypt the wrapped HPKE key is hard coded as kKMSUnwrapPath. We will either put the URL in KeyData::key_encryption_key_uri or use an environment variable
Implement telemetry's init_azure
Implement role_credentials_provider
Handle private_key_cache_ttl_seconds option.
Allow to specify key_id for private key API PrivateKeyFetchingRequest.
Separate azure_cpio_lib_inside_tee and azure_cpio_lib_outside_tee implementation. Current azure_cpio_lib_inside_tee uses fake attestation report silently when it's outside TEE.
Improve aci_attestation_lib library
- Add tests. We test them regularly with our internal CI, but we haven't written tests in a similar way as the existing code base.
- Follow the coding convention
- Attest to an ephemeral wrapping key as a runtime claim in report_data as a proof-of-possession of a private key that can unwrap the private HPKE key
Improve definition for Azure blob_storage_client_provider. We defined azure_platform for blob_storage_client_provider_select_lib using the GCP implementation under /src/gcp. We think it's not used in B&A services, but we added it just to avoid build errors when we run build_and_test_all_in_docker.
Add azure_platform for test_lib_cpio_provider
Fix license under cloud_initializer/src/azure/ directory. We copied the gcp implementation, but accidentally added the Microsoft copyright. We made this PR without the fix because testing is expensive with our internal test infrastructure at this moment. We'll fix them within this PR if there is other change that needs to be made before merging.

google-cla · 2024-01-23T16:14:59Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

takuro-sato · 2024-01-23T16:16:04Z

scp/cc/cpio/client_providers/auth_token_provider/src/BUILD.bazel

+            "//:azure_platform": [
+                "//scp/cc/cpio/client_providers/auth_token_provider/src/gcp:gcp_auth_token_provider_lib",
+            ],


We are not doing anything here, because auth_token_provider is not implemented yet.

takuro-sato · 2024-01-23T16:16:43Z

...lient_providers/private_key_fetcher_provider/src/azure/azure_private_key_fetcher_provider.cc

+  return SuccessExecutionResult();
+}
+
+ExecutionResult AzurePrivateKeyFetcherProvider::SignHttpRequest(


We are not doing anything here, because auth_token_provider is not implemented yet.

takuro-sato · 2024-01-23T16:17:08Z

scp/cc/cpio/client_providers/kms_client_provider/src/azure/azure_kms_client_provider.cc

+  http_context.callback = bind(&AzureKmsClientProvider::OnDecryptCallback,
+                               this, decrypt_context, _1);
+
+  auto execution_result = http_client_->PerformRequest(http_context);


We will use auth_token_provider for authentication once it's implemented.

takuro-sato · 2024-01-23T16:17:55Z

...c/cpio/client_providers/instance_client_provider/src/azure/azure_instance_client_provider.cc

This class currently returns fake values or FailureExecutionResult(SC_UNKNOWN). We will implement them in a future PR.

takuro-sato · 2024-01-23T16:18:20Z

scp/cc/cpio/client_providers/role_credentials_provider/src/BUILD.bazel

+            "//:azure_platform": [
+                "//scp/cc/cpio/client_providers/role_credentials_provider/src/gcp:gcp_role_credentials_provider_lib",
+            ],


This is because we haven't implemented our version. We will Azure our version in a future PR.

takuro-sato · 2024-01-23T16:18:48Z

...providers/private_key_fetcher_provider/src/azure/azure_private_key_fetcher_provider_utils.cc

+
+namespace google::scp::cpio::client_providers {
+
+void AzurePrivateKeyFetchingClientUtils::CreateHttpRequest(


We have TODOs for future PRs in this function:

Handle private_key_cache_ttl_seconds option.

Allow to specify key_id for private key API

takuro-sato · 2024-01-23T16:19:13Z

scp/cc/cpio/client_providers/kms_client_provider/src/BUILD.bazel

+                # We should split implementation here. Currently it uses fake attestation silently when it's outside TEE.
+                "//scp/cc/public/cpio/interface:azure_cpio_lib_inside_tee": [
+                    "//scp/cc/cpio/client_providers/kms_client_provider/src/azure:azure_kms_client_provider_lib",
+                ],
+                "//scp/cc/public/cpio/interface:azure_cpio_lib_outside_tee": [
+                    "//scp/cc/cpio/client_providers/kms_client_provider/src/azure:azure_kms_client_provider_lib",
+                ],


We will separate these two properly in a future PR. Current azure_cpio_lib_inside_tee uses fake attestation report silently when it's outside TEE.

takuro-sato · 2024-01-23T16:20:02Z

scp/cc/azure/attestation/BUILD.bazel

aci_attestation_lib has the following TODOs for future PRs:

Add tests. We test them regularly with our internal CI, but we haven't written tests in a similar way as the existing code base.

Follow the coding convention

Attest to an ephemeral wrapping key as a runtime claim in report_data as a proof-of-possession of a private key that can unwrap the private HPKE key

takuro-sato · 2024-01-23T16:20:27Z

scp/cc/cpio/client_providers/blob_storage_client_provider/src/BUILD.bazel

+            "//:azure_platform": [
+                "//scp/cc/cpio/client_providers/blob_storage_client_provider/src/gcp:gcp_blob_storage_client_provider_lib",
+            ],


We defined azure_platform for blob_storage_client_provider_select_lib using the GCP implementation under /src/gcp. We think it's not used in B&A services at this moment, but we added it just to avoid build errors when we run build_and_test_all_in_docker. We will find a better solution for that problem in a future PR.

takuro-sato · 2024-01-23T16:21:05Z

scp/cc/cpio/client_providers/cloud_initializer/src/azure/BUILD.bazel

@@ -0,0 +1,42 @@
+# Portions Copyright (c) Microsoft Corporation


We copied the gcp implementation, but accidentally added the Microsoft copyright. We made this PR without the fix because testing is expensive with our infrastructure at this moment. We'll fix them within this PR if there is other change that needs to be made before merging.

takuro-sato · 2024-01-23T16:21:26Z

scp/cc/cpio/client_providers/cloud_initializer/src/azure/no_op_initializer.cc

@@ -0,0 +1,46 @@
+/*
+ * Portions Copyright (c) Microsoft Corporation


We copied the gcp implementation, but accidentally added the Microsoft copyright. We made this PR without the fix because testing is expensive with our infrastructure at this moment. We'll fix them within this PR if there is other change that needs to be made before merging.

takuro-sato · 2024-01-23T16:21:35Z

scp/cc/cpio/client_providers/cloud_initializer/src/azure/no_op_initializer.h

@@ -0,0 +1,40 @@
+/*
+ * Portions Copyright (c) Microsoft Corporation


We copied the gcp implementation, but accidentally added the Microsoft copyright. We made this PR without the fix because testing is expensive with our infrastructure at this moment. We'll fix them within this PR if there is other change that needs to be made before merging.

Co-authored-by: Dominic Ayre <[email protected]> Co-authored-by: Joe Powell <[email protected]> Co-authored-by: Kapil Vaswani <[email protected]> Co-authored-by: Ken Gordon <[email protected]> Co-authored-by: Mahati Chamarthy <[email protected]> Co-authored-by: Ronny Bjones <[email protected]>

takuro-sato commented Jan 23, 2024

View reviewed changes

takuro-sato force-pushed the add-azure-support branch 2 times, most recently from b2c825f to bf865c7 Compare January 23, 2024 17:05

takuro-sato mentioned this pull request Jan 23, 2024

Add Azure support privacysandbox/bidding-auction-servers#9

Open

takuro-sato marked this pull request as ready for review January 23, 2024 17:11

takuro-sato force-pushed the add-azure-support branch 2 times, most recently from 10d0c87 to 46b4a32 Compare January 25, 2024 13:25

Fix bazel config for azure/attestation (#9)

4c83851

takuro-sato mentioned this pull request Jun 7, 2024

Add Azure support (for v3.4.0) KenGordon/data-plane-shared-libraries#24

Open

DomAyre mentioned this pull request Jun 10, 2024

Add Azure Support (in lieu of a PR) #7

Open

takuro-sato mentioned this pull request Jun 17, 2024

Fix copyright KenGordon/data-plane-shared-libraries#30

Merged

takuro-sato mentioned this pull request Sep 9, 2024

Azure support PR targeting 3e92e75 (used by bidding-auction-servers v3.10.0) #9

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Azure support #1

Add Azure support #1

takuro-sato commented Jan 23, 2024 •

edited

Loading

google-cla bot commented Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024

takuro-sato Jan 23, 2024


		namespace google::scp::cpio::client_providers {

		void AzurePrivateKeyFetchingClientUtils::CreateHttpRequest(

		@@ -0,0 +1,42 @@
		# Portions Copyright (c) Microsoft Corporation

		@@ -0,0 +1,46 @@
		/*
		* Portions Copyright (c) Microsoft Corporation

		@@ -0,0 +1,40 @@
		/*
		* Portions Copyright (c) Microsoft Corporation

Add Azure support #1

Are you sure you want to change the base?

Add Azure support #1

Conversation

takuro-sato commented Jan 23, 2024 • edited Loading

Adding Azure support

Changes

TODOs for future PRs

google-cla bot commented Jan 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

takuro-sato commented Jan 23, 2024 •

edited

Loading