Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dapr dotnet sample #61

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 16 additions & 0 deletions samples/dapr-pubsub-dotnet/build.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
param (
[Parameter(Mandatory=$True)]
[string]$Version

[Parameter(Mandatory=$True)]
[string]$ContainerRegistry
)

docker build .\src\TelemetryProcessor\TelemetryTransformer\. -f .\src\TelemetryProcessor\TelemetryTransformer\Dockerfile -t telemetrytransformer:$Version
docker build .\src\TelemetryProcessor\TelemetryPersister\. -f .\src\TelemetryProcessor\TelemetryPersister\Dockerfile -t telemetrypersister:$Version

docker tag telemetrypersister:$Version $ContainerRegistry/explore-iot-operations/samples/dapr-pubsub-dotnet/telemetrypersister:$Version
docker tag telemetrytransformer:$Version $ContainerRegistry/explore-iot-operations/samples/dapr-pubsub-dotnet/telemetrytransformer:$Version

docker push $ContainerRegistry/explore-iot-operations/samples/dapr-pubsub-dotnet/telemetrypersister:$Version
docker push $ContainerRegistry/explore-iot-operations/samples/dapr-pubsub-dotnet/telemetrytransformer:$Version
13 changes: 13 additions & 0 deletions samples/dapr-pubsub-dotnet/deploy.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
param (
[Parameter(Mandatory=$True)]
[string]$ContainerRegistry,

[Parameter(Mandatory=$True)]
[string]$Version
)

$contents = (Get-Content .\deploy\telemetryprocessor.yaml) -Replace '#{container_registry}#', $ContainerRegistry

$contents = $contents -replace '#{image_version}#', $Version

$contents | kubectl apply -n azure-iot-operations -f -
202 changes: 202 additions & 0 deletions samples/dapr-pubsub-dotnet/deploy/telemetryprocessor.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: aio-mq-pubsub
namespace: azure-iot-operations
spec:
type: pubsub.aio-mq-pubsub-pluggable
version: v1
metadata:
- name: url
value: "aio-mq-dmqtt-frontend:8883"
- name: satTokenPath
value: "/var/run/secrets/tokens/mqtt-client-token"
- name: tlsEnabled
value: true
- name: caCertPath
value: "/var/run/certs/aio-mq-ca-cert/ca.crt"
- name: logLevel
value: "Info"
---
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: aio-mq-statestore
namespace: azure-iot-operations
spec:
type: state.aio-mq-statestore-pluggable
version: v1
metadata:
- name: url
value: "aio-mq-dmqtt-frontend:8883"
- name: satTokenPath
value: "/var/run/secrets/tokens/mqtt-client-token"
- name: tlsEnabled
value: true
- name: caCertPath
value: "/var/run/certs/aio-mq-ca-cert/ca.crt"
- name: logLevel
value: "Info"
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: dapr-client
namespace: azure-iot-operations
annotations:
aio-mq-broker-auth/group: dapr-workload
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: telemetrytransformer
namespace: azure-iot-operations
labels:
app: telemetrytransformer
spec:
replicas: 1
selector:
matchLabels:
app: telemetrytransformer
template:
metadata:
labels:
app: telemetrytransformer
annotations:
dapr.io/enabled: "true"
dapr.io/unix-domain-socket-path: "/tmp/dapr-components-sockets"
dapr.io/app-id: "telemetrytransformer"
dapr.io/enable-api-logging: "true"
dapr.io/app-port: "5050"
dapr.io/app-protocol: "grpc"
dapr.io/log-level: "debug"
dapr.io/sidecar-liveness-probe-delay-seconds: "30"
dapr.io/sidecar-liveness-probe-timeout-seconds: "10"
spec:
serviceAccountName: dapr-client
volumes:
- name: dapr-unix-domain-socket
emptyDir: {}
- name: mqtt-client-token
projected:
sources:
- serviceAccountToken:
path: mqtt-client-token
audience: aio-mq
expirationSeconds: 86400
- name: aio-ca-trust-bundle
configMap:
name: aio-ca-trust-bundle-test-only
imagePullSecrets:
- name: aio-pullsecret
containers:
- name: telemetrytransformer
image: #{container_registry}#/explore-iot-operations/samples/dapr-pubsub-dotnet/telemetrytransformer:#{image_version}#
imagePullPolicy: Always
env:
- name: Logging__LogLevel__Default
value: Debug
# Container for the pluggable component
- name: aio-mq-components
image: ghcr.io/azure/iot-mq-dapr-components:latest
volumeMounts:
- name: dapr-unix-domain-socket
mountPath: /tmp/dapr-components-sockets
- name: mqtt-client-token
mountPath: /var/run/secrets/tokens
- name: aio-ca-trust-bundle
mountPath: /var/run/certs/aio-mq-ca-cert/
---
kind: Service
apiVersion: v1
metadata:
name: telemetrytransformer-workload-svc
namespace: azure-iot-operations
labels:
app: telemetrytransformer
spec:
selector:
app: telemetrytransformer
ports:
- protocol: TCP
port: 5050 #6001
targetPort: 5050 # 6001
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: telemetrypersister
namespace: azure-iot-operations
labels:
app: telemetrypersister
spec:
replicas: 1
selector:
matchLabels:
app: telemetrypersister
template:
metadata:
labels:
app: telemetrypersister
annotations:
dapr.io/enabled: "true"
dapr.io/unix-domain-socket-path: "/tmp/dapr-components-sockets"
dapr.io/app-id: "telemetrypersister"
dapr.io/enable-api-logging: "true"
dapr.io/app-port: "80" #"6001"
dapr.io/app-protocol: "http"
dapr.io/log-level: "debug"
dapr.io/sidecar-liveness-probe-delay-seconds: "30"
dapr.io/sidecar-liveness-probe-timeout-seconds: "10"
spec:
serviceAccountName: dapr-client
volumes:
- name: dapr-unix-domain-socket
emptyDir: {}
- name: mqtt-client-token
projected:
sources:
- serviceAccountToken:
path: mqtt-client-token
audience: aio-mq
expirationSeconds: 86400
- name: aio-ca-trust-bundle
configMap:
name: aio-ca-trust-bundle-test-only
imagePullSecrets:
- name: aio-pullsecret
containers:
- name: telemetrypersister
image: #{container_registry}#/explore-iot-operations/samples/dapr-pubsub-dotnet/telemetrypersister:#{image_version}#
imagePullPolicy: Always
env:
- name: Logging__LogLevel__Default
value: Debug
# Container for the pluggable component
- name: aio-mq-components
image: ghcr.io/azure/iot-mq-dapr-components:latest
volumeMounts:
- name: dapr-unix-domain-socket
mountPath: /tmp/dapr-components-sockets
- name: mqtt-client-token
mountPath: /var/run/secrets/tokens
- name: aio-ca-trust-bundle
mountPath: /var/run/certs/aio-mq-ca-cert/
---
kind: Service
apiVersion: v1
metadata:
name: telemetrypersister-workload-svc
namespace: azure-iot-operations
labels:
app: telemetrypersister
spec:
selector:
app: telemetrypersister
ports:
- protocol: TCP
port: 80 #6001
targetPort: 80 # 6001
type: ClusterIP

120 changes: 120 additions & 0 deletions samples/dapr-pubsub-dotnet/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
## Introduction

This sample shows how you can use Dapr in .NET projects to receive messages from an MQTT topic and publish messages to another MQTT topic.

## Prerequisites

- [Docker Desktop](https://www.docker.com/products/docker-desktop/) or [Rancher Desktop](https://rancherdesktop.io/) to be able to build the container images on your computer
- [Powershell](https://learn.microsoft.com/en-us/powershell/scripting/install/installing-powershell?view=powershell-7.3) installed on your computer to be able to use the build and deploy powershell scripts
- A Kubernetes cluster where Dapr and Azure IoT Operations is installed
- A Container Registry where docker images can be pushed to
- A pullsecret for the Container Registry must exist in the Kubernetes cluster. The pullsecret must have the name `aio-pullsecret`. Information on how to create a pull secret for an Azure Container Registry can be found [here](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-auth-kubernetes#create-an-image-pull-secret). More generic information on how to create a pull secret can be found [here](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/).

## Overview

This sample consists of two .NET workloads that are communicating with each other via the IoT MQ broker.

![Solution overview](.static/img/overview.png)

- Raw JSON messages are published to the IoT MQ MQTT broker
- The `telemetrytransformer` component is implemented as a Console Application that uses Dapr pub/sub to receive (non cloud-event) raw messages from the IoT MQ broker
- The `telemetrytransformer` publishes cloud-event messages to the IoT MQ broker
- The `telemetrypersister` component is implemented as an ASP.NET Web API project that uses Dapr pub/sub to receive cloud-event messages from the IOT MQ broker
- The `telemetrypersister` has an additional subscription where Dapr is used to receive non cloud-event messages from an MQTT topic.

## How to build and deploy

- Build the .NET components using the `build.ps1` powershell script file.
This script takes 2 parameters:
- Version: this parameter is used to assign a tag to the container images that are being build for the `telemetrytransformer` and `telemetrypersister` component.
- ContainerRegistry: the URL of the container registry to where the images must be pushed.

- Deploy the sample to your Kubernetes cluster by using the `deploy.ps1` powershell script.
This script takes 3 parameters:
- ContainerRegistry: the URL of the container registry where the images reside.
- Version: the version of the `telemetrytransformer` and `telemetrypersister` components that you want to deploy

## Test the sample

You can test the sample by publishing JSON messages to the `devices/+/#` MQTT topic on the IOT MQ broker.

To do this, you first need to find out the external IP address that you can use to send messages to IoT MQ. You need the IP address of the IoT MQ aio-mq-dmqtt-frontend service.

To do this, execute the `kubectl get svc -n azure-iot-operations` command.
This command will show all the services that exist in the namespace. One of those services is the `aio-mq-dmqtt-frontend` service. Use the external IP address of that service in the following command:

> [!NOTE]
> The mosquitto pub needs to switch to a different auth, or we need to enable username/password. Maybe switch this to a mqtt client deployed onto the cluster similar to other tutorials

```
mosquitto_pub -h <dmqtt-frontend-ip> -t "devices/device1/sensor1" -m '{"Tag":"fan_speed","Value":35,"Timestamp":"2023-07-03T18:20:00Z"}' -q 1 -u client1 -P password
```

After you've published a message via the command above, you should see some information in the logs of the `telemetrytransformer` pod. For instance:

To get the name of the pod for which to retrieve the logs, execute this command:

```
kubectl get pods -n azure-iot-operations
```

```
kubectl logs -n azure-iot-operations telemetrytransformer-75cf9f8978-47fqm
```
shows:
```
<6>2023-07-04 07:54:17.711 +00:00 TelemetryTransformer.Services.DeviceTelemetryReceiver[0] OnTopicEvent called on topic devices/device1/sensor2
<6>2023-07-04 07:54:17.711 +00:00 TelemetryTransformer.Services.DeviceTelemetryReceiver[0] payload = {"Tag":"fan_speed","Value":37,"Timestamp":"2023-07-04T18:20:00Z"}
Received message: {"Tag":"fan_speed","Value":37,"Timestamp":"2023-07-04T18:20:00Z"}
Message deserialized:
Timestamp = 07/04/2023 18:20:00 +00:00
Tag = fan_speed
Value = 37
<6>2023-07-04 07:54:17.712 +00:00 TelemetryTransformer.Services.DeviceTelemetryReceiver[0] Publishing message ...
<6>2023-07-04 07:54:17.760 +00:00 TelemetryTransformer.Services.DeviceTelemetryReceiver[0] Message published.
```

Since the `telemetrytransformer` worker has published a new message to the IoT MQ broker, and the `telemetrypersister` worker is listening to the specific topic where the message has been sent to, you should see that the `telemetrypersister` has done some work as well.

Displaying the logs of the `telemetrypersister` should show this:

Again, first get the exact name of the pod by executing this command:

```
kubectl get pods -n azure-iot-operations
```

Once you have the name of the pod, execute this command:

```
kubectl logs -n azure-iot-operations telemetrypersister-7fcf59885d-fd9qb
```

```
<6>2023-07-04 07:54:17.784 +00:00 TelemetryPersister.Controllers.DeviceTelemetryController[0] DeviceTelemetry message received
<6>2023-07-04 07:54:17.784 +00:00 TelemetryPersister.Controllers.DeviceTelemetryController[0] Persisting telemetry for device: {"deviceId":"device1","timestamp":"2023-07-04T18:20:00+00:00","tag":"fan_speed","value":37}
```

The `telemetrypersister` also contains a subscription on a topic where raw payloads (no cloudevents) are expected.
This subscription is listening on the `commands/#` topic.

To test this, publish a JSON message to the `commands/#` topic in the IoT MQ broker:

> [!NOTE]
> The mosquitto pub needs to switch to a different auth, or we need to enable username/password. Maybe switch this to a mqtt client deployed onto the cluster similar to other tutorials

```
mosquitto_pub -h <dmqtt-frontend-ip> -t "commands/exec" -m '{"cmd":"reboot"}' -q 1 -u client1 -P password
```

The logs of the `telemetrypersister` should show this:

```
<6>2023-07-04 14:14:56.860 +00:00 TelemetryPersister.Controllers.DeviceTelemetryController[0] Command received: reboot
```

> Be aware that in this sample, all loglevels are set to 'Debug'. This means a lot of logs are generated, which is obviously not something you want to do by default in production scenarios!

## Acknowledgements

Big thanks to [Frederik Gheysels](https://github.com/fgheysels) for contributing this sample!
Loading