Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor samples with terraform and defaultCredential #133

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
893b7ed
refactor for mutiple-property-source
backwind1233 Dec 28, 2021
55f122b
refactor for servicebus samples
backwind1233 Dec 28, 2021
a788728
updated
backwind1233 Dec 28, 2021
690cadc
updated servicebus
backwind1233 Dec 28, 2021
afd138b
middle stage,just save code
backwind1233 Dec 28, 2021
f9bf167
add middle stage code
backwind1233 Dec 30, 2021
bbd2000
update storage queue integration sample
backwind1233 Dec 30, 2021
06a0ceb
add middle stage
backwind1233 Dec 30, 2021
db77f21
fix eh-kafka and sb-multi-binders sample errors
backwind1233 Dec 30, 2021
7207c0b
update variables.tf
backwind1233 Dec 30, 2021
0a65f2a
update variables.tf
backwind1233 Dec 30, 2021
bcd661c
format code
backwind1233 Dec 30, 2021
af6df66
format code
backwind1233 Dec 30, 2021
ea9cec4
update value names in tf scripts
backwind1233 Dec 30, 2021
291e288
add .
backwind1233 Dec 30, 2021
2f8fd68
check azurecaf_name names
backwind1233 Dec 30, 2021
62ea3cf
check output value case
backwind1233 Dec 30, 2021
1ab3f71
check description
backwind1233 Dec 30, 2021
2ac7241
remove sensitive = true
backwind1233 Dec 30, 2021
4409e19
rename azurerm_role_assignment
backwind1233 Dec 30, 2021
37be5ff
add sensitive = true
backwind1233 Dec 31, 2021
78e80e4
- add log for eventhubs-kafka
backwind1233 Jan 3, 2022
7cbf643
- add log for eventhubs-binder
backwind1233 Jan 3, 2022
eb05f87
- add log for eventhubs-multibinders
backwind1233 Jan 3, 2022
ddfe4a2
- add log for servicebus-jms-queue
backwind1233 Jan 3, 2022
58ed1c8
add template
backwind1233 Jan 3, 2022
fbdd99a
add log for servicebus jms topic
backwind1233 Jan 3, 2022
7757b97
add log for servicebus multibinders
backwind1233 Jan 3, 2022
bb1e4b5
add log for servicebus queue binder
backwind1233 Jan 3, 2022
16f8858
add log for servicebus topic binder
backwind1233 Jan 3, 2022
3fb1572
add log for storage-queue
backwind1233 Jan 3, 2022
361735c
add log for multiple-property-source
backwind1233 Jan 3, 2022
bf0ea1a
remove checklist
backwind1233 Jan 3, 2022
2eff124
update readme: spring-cloud-azure-sample-eventhubs-kafka
backwind1233 Jan 3, 2022
90e229a
update readme: eventhubs-integration
backwind1233 Jan 3, 2022
e62bcaf
update readme: eventhubs-integration
backwind1233 Jan 3, 2022
511bf7f
update readme: eventhubs-binder
backwind1233 Jan 3, 2022
3e2e9ac
update readme: eventhubs-multibinders
backwind1233 Jan 3, 2022
a8faa19
update readme: multiple-property-source
backwind1233 Jan 3, 2022
0149ef0
update readme: servicebus-jms-queue
backwind1233 Jan 3, 2022
5d88076
update readme: servicebus-jms-topic
backwind1233 Jan 3, 2022
7b0962a
update readme: servicebus-multibinders
backwind1233 Jan 3, 2022
6a7b657
update readme: servicebus-queue-binder
backwind1233 Jan 3, 2022
b5b30b5
update readme: servicebus-topic-binder
backwind1233 Jan 3, 2022
f30f92a
update readme: storage-queue-integration
backwind1233 Jan 3, 2022
b2d0a05
update readme: storage-queue-operation
backwind1233 Jan 3, 2022
6a36187
update readme: storage-queue-operation
backwind1233 Jan 3, 2022
522fb02
update readme: servicebus-topic-binder
backwind1233 Jan 3, 2022
ea2da9d
update readme: servicebus-queue-binder
backwind1233 Jan 3, 2022
9a5950a
update readme: servicebus-multibinders
backwind1233 Jan 3, 2022
3cd5b20
update readmes
backwind1233 Jan 3, 2022
4078eda
update readme
backwind1233 Jan 3, 2022
b928460
update readme for ## Verify This Sample
backwind1233 Jan 3, 2022
015ce14
update pricing-tier
backwind1233 Jan 3, 2022
4a9c88d
update topic-client-id
backwind1233 Jan 3, 2022
24571e0
fix @TODO
backwind1233 Jan 3, 2022
0d0a5bc
remove application-*.yaml
backwind1233 Jan 4, 2022
2985fe9
remove application-*.yaml
backwind1233 Jan 4, 2022
a45c96c
reformt tf scripts
backwind1233 Jan 4, 2022
c840574
the max length of "azurerm_key_vault" is 24
backwind1233 Jan 4, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
variable "application_name" {
type = string
description = "The name of your application"
description = "The name of your application."
default = "cosmos-sample"
}

variable "location" {
type = string
description = "The Azure region where all resources in this example should be created"
description = "The Azure region where all resources in this example should be created."
default = "eastus"
}

variable "sample_tag_value" {
type = string
description = "The value of spring-cloud-azure-sample tag"
description = "The value of spring-cloud-azure-sample tag."
default = "true"
}
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ mvn clean spring-boot:run

## Verify This Sample

Verify in your app’s logs that a similar message was posted:
Verify in your app’s logs that similar messages were posted:
```shell

...
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ output "object_id" {
}

output "cosmos_database_name" {
value = var.cosmos_database_name
value = var.cosmos_database_name
}


Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
variable "application_name" {
type = string
description = "The name of your application"
description = "The name of your application."
default = "data-cosmos-sample"
}

variable "location" {
type = string
description = "The Azure region where all resources in this example should be created"
description = "The Azure region where all resources in this example should be created."
default = "eastus"
}

Expand All @@ -18,6 +18,6 @@ variable "cosmos_database_name" {

variable "sample_tag_value" {
type = string
description = "The value of spring-cloud-azure-sample tag"
description = "The value of spring-cloud-azure-sample tag."
default = "true"
}
Original file line number Diff line number Diff line change
@@ -1,158 +1,160 @@
# Spring Cloud Azure Starter Integration for Event Hubs Code Sample shared library for Java

## Key concepts

This sample demonstrates how to use `Spring Integration` for `Azure
Event Hubs`.

## What You Will Build
You will build an application to send and receive messages for Event Hubs using Spring Integration.

## What You Need

- [An Azure subscription](https://azure.microsoft.com/free/)
- [Terraform](https://www.terraform.io/)
- [Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli)
- [JDK8](https://www.oracle.com/java/technologies/downloads/) or later
- Maven
- You can also import the code straight into your IDE:
- [IntelliJ IDEA](https://www.jetbrains.com/idea/download)

## Provision Azure Resources Required to Run This Sample
This sample will create Azure resources using Terraform. If you choose to run it without using Terraform to provision resources, please pay attention to:
> [!IMPORTANT]
> If you choose to use a security principal to authenticate and authorize with Azure Active Directory for accessing an Azure resource
> please refer to [Authorize access with Azure AD](https://microsoft.github.io/spring-cloud-azure/docs/current/reference/html/index.html#authorize-access-with-azure-active-directory) to make sure the security principal has been granted the sufficient permission to access the Azure resource.

### Authenticate Using the Azure CLI
Terraform must authenticate to Azure to create infrastructure.

In your terminal, use the Azure CLI tool to setup your account permissions locally.

```shell
az login
```

Your browser window will open and you will be prompted to enter your Azure login credentials. After successful authentication, your terminal will display your subscription information. You do not need to save this output as it is saved in your system for Terraform to use.

```shell
You have logged in. Now let us find all the subscriptions to which you have access...

[
{
"cloudName": "AzureCloud",
"homeTenantId": "home-Tenant-Id",
"id": "subscription-id",
"isDefault": true,
"managedByTenants": [],
"name": "Subscription-Name",
"state": "Enabled",
"tenantId": "0envbwi39-TenantId",
"user": {
"name": "[email protected]",
"type": "user"
}
}
]
```

If you have more than one subscription, specify the subscription-id you want to use with command below:
```shell
az account set --subscription <your-subscription-id>
```

### Provision the Resources

After login Azure CLI with your account, now you can use the terraform script to create Azure Resources.

```shell
# In the root directory of the sample
# Initialize your Terraform configuration
terraform -chdir=./terraform init

# Apply your Terraform Configuration
# Type `yes` at the confirmation prompt to proceed.
terraform -chdir=./terraform apply

```




It may take a few minutes to run the script. After successful running, you will see prompt information like below:

```shell


azurerm_resource_group.main: Creating...
azurerm_resource_group.main: Creation complete after 3s ...
azurerm_storage_account.storage_account: Creating...
azurerm_eventhub_namespace.eventhubs_namespace: Still creating... [10s elapsed]
...
azurerm_storage_account.storage_account: Creation complete after 38s ...
azurerm_storage_container.storage_container: Creating...
azurerm_role_assignment.role_storage_account_contributor: Creating...
azurerm_storage_container.storage_container: Creation complete after 1s ...
azurerm_role_assignment.role_storage_blob_data_owner: Creating...
...
azurerm_role_assignment.role_storage_blob_data_owner: Creation complete after 25s ...
azurerm_role_assignment.role_storage_account_contributor: Creation complete after 29s ...
...
azurerm_eventhub_namespace.eventhubs_namespace: Creation complete after 1m23s ...
azurerm_eventhub.eventhubs: Creating...
azurerm_eventhub.eventhubs: Creation complete after 7s ...
azurerm_role_assignment.role_eventhubs_data_owner: Creating...
...
azurerm_role_assignment.role_eventhubs_data_owner: Creation complete after 24s ...

Apply complete! Resources: 8 added, 0 changed, 0 destroyed.

Outputs:
...

```

## Getting started

Running this sample will be charged by Azure. You can check the usage and bill at
[this link][azure-account].


### Create Azure resources

1. Create [Azure Event Hubs][create-event-hubs].
After creating the Azure Event Hubs, you
can create your own Consumer Group or use the default "$Default" Consumer Group.

2. Create [Azure Storage][create-azure-storage] for checkpoint use.

### Configuration credential options
You can go to [Azure portal](https://ms.portal.azure.com/) in your web browser to check the resources you created.

We have several ways to config the Spring Integration for Event Hubs. You can choose anyone of them.
### Export Output to Your Local Environment
Running the command below to export environment values:

>[!Important]
>
> When using the Restful API to send messages, the **Active profiles** must contain `manual`.
>
#### Method 1: Connection string based usage
```shell
source ./terraform/setup_env.sh
```

1. Update [application.yaml][application.yaml].
```yaml
spring:
cloud:
azure:
eventhubs:
connection-string: ${AZURE_EVENTHUBS_CONNECTION_STRING}
processor:
checkpoint-store:
container-name: ${AZURE_STORAGE_CONTAINER_NAME}
account-name: ${AZURE_STORAGE_ACCOUNT_NAME}
account-key: ${AZURE_STORAGE_ACCOUNT_KEY}
```
## Run Locally

1. Update event hub name and consumer group in
[ReceiveController][receive-controller] and [SendController][send-controller].
In your terminal, run `mvn clean spring-boot:run`.

#### Method 2: Service principal based usage

1. Create a service principal for use in by your app. Please follow
[create service principal from Azure CLI][create-sp-using-azure-cli].
```shell
mvn clean spring-boot:run
```

2. Add Role Assignment for Event Hubs. See
[Service principal for Azure resources with Event Hubs][role-assignment]
to add role assignment for Event Hubs. Assign `Contributor` role for event hubs.
## Verify This Sample

3. Update [application-sp.yaml][application-sp.yaml].
```yaml
spring:
cloud:
azure:
profile:
tenant-id: ${AZURE_TENANT_ID}
credential:
client-id: ${AZURE_CLIENT_ID}
client-secret: ${AZURE_CLIENT_SECRET}
eventhubs:
namespace: ${AZURE_EVENTHUBS_NAMESPACE}
processor:
checkpoint-store:
container-name: ${AZURE_STORAGE_CONTAINER_NAME}
account-name: ${AZURE_STORAGE_ACCOUNT_NAME}
```
> We should specify `spring.profiles.active=sp` to run the Spring Boot application.
For App Service, please add a configuration entry for this.
#### Method 3: MSI credential based usage
1. Send a POST request

##### Set up managed identity
```shell
curl -X POST http://localhost:8080/messages?message=hello
```

Please follow [create managed identity][create-managed-identity] to set up managed identity.

##### Add Role Assignment for Event Hubs
2. Verify in your app’s logs that similar messages were posted:

1. See [Managed identities for Azure resources with Event Hubs][role-assignment]
to add role assignment for Event Hubs. Assign `Contributor` role for managed identity.
```shell
New message received: 'hello'
Message 'hello' successfully checkpointed
```


## Clean Up Resources
After running the sample, if you don't want to run the sample, remember to destroy the Azure resources you created to avoid unnecessary billing.

##### Update MSI related properties
The terraform destroy command terminates resources managed by your Terraform project.
To destroy the resources you created.

1. Update [application-mi.yaml][application-mi.yaml].
```yaml
spring:
cloud:
azure:
credential:
managed-identity-client-id: ${AZURE_MANAGED_IDENTITY_CLIENT_ID}
profile:
tenant-id: ${AZURE_TENANT_ID}
eventhubs:
namespace: ${AZURE_EVENTHUBS_NAMESPACE}
processor:
checkpoint-store:
container-name: ${AZURE_STORAGE_CONTAINER_NAME}
account-name: ${AZURE_STORAGE_ACCOUNT_NAME}
```
> We should specify `spring.profiles.active=mi` to run the Spring Boot application.
For App Service, please add a configuration entry for this.
```shell
terraform -chdir=./terraform destroy
```

##### Redeploy Application

If you update the `spring.cloud.azure.credential.managed-identity-client-id`
property after deploying the app, or update the role assignment for
services, please try to redeploy the app again.

> You can follow
> [Deploy a Spring Boot JAR file to Azure App Service][deploy-spring-boot-application-to-app-service]
> to deploy this application to App Service

## Examples

1. Run the `mvn spring-boot:run` in the root of the code sample to get the app running.

2. Send a POST request

$ ~~curl -X POST http://localhost:8080/messages?message=hello~~

3. Verify in your app’s logs that a similar message was posted:

New message received: 'hello'
Message 'hello' successfully checkpointed

4. Delete the resources on [Azure Portal][azure-portal] to avoid unexpected charges.


## Troubleshooting

## Next steps

## Contributing


<!-- LINKS -->
[azure-account]: https://azure.microsoft.com/account/
[azure-portal]: https://ms.portal.azure.com/
[create-event-hubs]: https://docs.microsoft.com/azure/event-hubs/
[create-azure-storage]: https://docs.microsoft.com/azure/storage/
[create-managed-identity]: https://github.com/Azure-Samples/azure-spring-boot-samples/blob/spring-cloud-azure_4.0/create-managed-identity.md
[create-sp-using-azure-cli]: https://github.com/Azure-Samples/azure-spring-boot-samples/blob/spring-cloud-azure_4.0/create-sp-using-azure-cli.md
[eventhub-operation]: https://github.com/Azure/azure-sdk-for-java/blob/azure-spring-boot_3.6.0/sdk/spring/azure-spring-integration-eventhubs/src/spring-cloud-azure_4.0/java/com/azure/spring/integration/eventhub/api/EventHubOperation.java
[receive-controller]: src/main/java/com/azure/spring/sample/eventhubs/ReceiveController.java
[send-controller]: src/main/java/com/azure/spring/sample/eventhubs/SendController.java
[application.yaml]: src/main/resources/application.yaml
[application-sp.yaml]: src/main/resources/application-sp.yaml
[application-mi.yaml]: src/main/resources/application-mi.yaml
[deploy-spring-boot-application-to-app-service]: https://docs.microsoft.com/java/azure/spring-framework/deploy-spring-boot-java-app-with-maven-plugin?toc=%2Fazure%2Fapp-service%2Fcontainers%2Ftoc.json&view=azure-java-stable

This file was deleted.

This file was deleted.

Loading