[IDP-929] Add support for the pull delivery model (#42)

* Add support for the pull model * Add release method * Add new methods * Added test validaiton for reject & release cloud events * Use type alias * Initial refactor and create pull handlers * Refactor according to feedback * Readme changes * Readme changes and ci changes * Upgrade project to .NET 8 * Upgrade docker .net version to 8 * Remove redundant adduser command from dockerfile * Refactor according to feedback * Refactor readme for more clarity regarding topic support * Undo appsettings changes * Readme update --------- Co-authored-by: He Qian Wang <[email protected]>
workleap · Mar 11, 2024 · 181e827 · 181e827
1 parent 716abe4
commit 181e827
Show file tree

Hide file tree

Showing 24 changed files with 583 additions and 101 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -2,7 +2,7 @@ name: CI
 
 on:
   pull_request:
-    branches: ["main", "master"]
+    branches: ["main"]
     paths-ignore: ["*.md"]
 
 jobs:
@@ -13,12 +13,14 @@ jobs:
       - name: Checkout repository
         uses: actions/checkout@v4
 
+      - uses: actions/setup-dotnet@v4
+
       - name: Docker metadata
         id: meta
         uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81 # v5
         with:
           images: workleap/eventgridemulator
-          
+
       - name: Docker build
         uses: docker/build-push-action@4a13e500e55cf31b7a5d59a38ab2040ab0f42f56 # v5
         with:

diff --git a/README.md b/README.md
@@ -1,13 +1,16 @@
 # Workleap Azure Event Grid Emulator
 
-This is an open source emulator for [Azure Event Grid](https://learn.microsoft.com/en-us/azure/event-grid/overview) that supports only the [push delivery](https://learn.microsoft.com/en-us/azure/event-grid/push-delivery-overview) model. Based on ASP.NET Core, this emulator provides a cross-platform experience for developers wanting to try Azure Event Grid easily in a local environment without having to deploy to Azure.
+This is an open source emulator for [Azure Event Grid](https://learn.microsoft.com/en-us/azure/event-grid/overview) that supports both the [push delivery](https://learn.microsoft.com/en-us/azure/event-grid/push-delivery-overview) and the [pull delivery](https://learn.microsoft.com/en-us/azure/event-grid/pull-delivery-overview) models. Based on ASP.NET Core, this emulator provides a cross-platform experience for developers wanting to try Azure Event Grid easily in a local environment without having to deploy to Azure.
+
+The emulator supports two Events delivery formats: Push and Pull.
 
 This project is not affiliated, associated, authorized, endorsed by, or in any way officially connected with Microsoft.
 
 ## Features
-
-- Support for multiple Event Grid topics by sending events to `http://127.0.0.1:6500/<topic-name>/api/events`.
+- Support publishing events to Custom Topics (EventGridEvents/CloudEvents) and Namespace Topics (only CloudEvents)
+- Support Push & Pull Delivery Models
 - Push delivery to configured webhooks defined in the emulator configuration file (more details below).
+- Pull delivery API client commands supported in the emulator (more details below).
 - Simple but durable message delivery and retry based on the [Azure Event Grid documentation](https://learn.microsoft.com/en-us/azure/event-grid/delivery-and-retry).
 - Ability to add and remove topics and webhooks at runtime without having to restart the emulator.
 - As the emulator is built on top of ASP.NET Core, you can follow this [Microsoft documentation](https://learn.microsoft.com/en-us/aspnet/core/security/docker-compose-https) to run on HTTPS.
@@ -21,7 +24,7 @@ You must have [Docker](https://www.docker.com/get-started/) installed. This Even
 The first step is to **create a configuration file** for the emulator to know the topics, and for each topic, the webhooks to call when an event is published.
 Create a configuration file named `appsettings.json` somewhere on your computer, for instance: `C:\eventgridemulator\appsettings.json`.
 
-It should look like this:
+### Push Delivery configuration
 
 ```json
 {
@@ -31,13 +34,26 @@ It should look like this:
       "http://host.docker.internal:7221/eventgrid"
     ],
     "topic2": [
-      "https://mydockercontainer:5122/eventgrid/domainevents",
+      "https://mydockercontainer:5122/eventgrid/domainevents"
+    ],
+  }
+}
+```
+In the example for push delivery, we have two topics, `topic1` and `topic2`. If an event is sent to the emulator on this URL `http://127.0.0.1:6500/topic1/api/events`, the emulator would forward the events to `https://host.docker.internal:5122/my-webhook` and `http://host.docker.internal:7221/eventgrid` on your host machine. As the emulator runs on Docker, you must use the `host.docker.internal` (emulator must make an http ) host whenever you want to call a webhook on your host machine.
+
+### Pull delivery configuration
+```json
+{
+  "Topics": {
+    "topicfoobar": [
+      "pull://foo-subscription",
+      "pull://bar-subscription"
     ]
   }
 }
 ```
 
-In this example, we have two topics, `topic1` and `topic2`. If an event is sent to the emulator on this URL `http://127.0.0.1:6500/topic1/api/events`, the emulator would forward the events to `https://host.docker.internal:5122/my-webhook` and `http://host.docker.internal:7221/eventgrid` on your host machine. As the emulator runs on Docker, you must use the `host.docker.internal` host whenever you want to call a webhook on your host machine.
+In the example for pull delivery, we have a topics, `topicfoobar`. If an event is sent to the emulator on this URL `http://127.0.0.1:6500/topics/topicfoobar:publish`, the emulator would make the events available to pull at `pull://foo-subscription` and `pull://bar-subscription` on your host machine.
 
 **Run the Event Grid emulator with docker run**
 
@@ -67,9 +83,9 @@ services:
 
 From the directory in which the file resides, run the `docker compose up` command.
 
-**Sending and receiving events**
+## Publish and Receive Events using Push Delivery for Custom Topic
 
-Now that the emulator is running, you can send events to it and receive them in your webhooks. If you're using C#, follow [these steps from the Microsoft documentation](https://learn.microsoft.com/en-us/dotnet/api/overview/azure/messaging.eventgrid-readme?view=azure-dotnet):
+Push Delivery: Now that the emulator is running, you can send both EventGridEvents and CloudEvents to the endpoint and receive them in your webhooks. If you're using C#, follow [these steps from the Microsoft documentation](https://learn.microsoft.com/en-us/dotnet/api/overview/azure/messaging.eventgrid-readme?view=azure-dotnet):
 
 ```csharp
 // Change "my-topic" to the name of your topic.
@@ -78,6 +94,57 @@ Now that the emulator is running, you can send events to it and receive them in
 var client = new EventGridPublisherClient(
     new Uri("http://127.0.0.1:6500/my-topic/api/events"),
     new AzureKeyCredential("fakeAccessKey"));
+
+// Create and send a CloudEvent to EventGrid
+var cloudEvent = new new CloudEvent("<source>", "<type>", data);
+await client.SendEventAsync(cloudEvent);
+
+// Create and send an EventGridEvent to EventGrid
+var eventGridEvent = new EventGridEvent(
+    subject: "<source>",
+    eventType: "<type>",
+    dataVersion: "<version>",
+    data: data);
+await client.SendEventAsync(eventGridEvent);
+
+// An url with the correct url would need to be exposed to process the push delivery events
+[HttpPost("<my-topic endpoint defined in config>")]
+public IActionResult Post([FromBody]EventGridEvent[] value)
+{
+...
+}
+
+## Publish and Receive Events with Pull Delivery Model for Namespace Topic
+
+```
+Pull Delivery: Once the emulator is running, we can send CloudEvents to the endpoint and pull/acknowledge events with api calls.
+
+We support the following Queue delivery APIs:
+
+- `PublishCloudEventsAsync`: publishes an event from the queue.
+- `ReceiveCloudEventsAsync`: receives an event from the queue.
+- `AcknowledgeCloudEventsAsync`: acknowledges that the received event is processed successfully and delete from the queue.
+- `ReleaseCloudEventsAsync`: releases the received event and requeues the event.
+- `RejectCloudEventsAsync`: rejects the received event and delete from the queue.
+
+
+```csharp
+// The authentication mechanism is actually ignored by the emulator.
+// If you must provide a TokenCredential instead of an access key, the emulator must be running on HTTPS.
+var client = new EventGridClient(
+    new Uri("http://127.0.0.1:6500"),
+    new AzureKeyCredential("fakeAccessKey"));
+
+// Example of how to can publish an event with EventGridClient
+client.PublishCloudEventsAsync(topicName, new CloudEvent("<source>", "<type>", data));
+
+// Example of how to receive an event with EventGridClient
+var events = await client.ReceiveCloudEventsAsync(topicName, eventSubscriptionName);
+
+//  Example of how to can acknowledge an event from queue
+// Reject/Release would be the same format except with RejectCloudEventsAsync/RejectOptions and ReleaseCloudEventsAsync/ReleaseOptions
+await client.AcknowledgeCloudEventsAsync(topicName, eventSubscriptionName, new AcknowledgeOptions([<lock token obtained from the received cloud event>]));
+
 ```
 
 ## Additional information

diff --git a/global.json b/global.json
@@ -1,6 +1,6 @@
 {
   "sdk": {
-    "version": "7.0.406",
+    "version": "8.0.201",
     "rollForward": "latestMinor",
     "allowPrerelease": false
   }

diff --git a/src/.editorconfig b/src/.editorconfig
@@ -157,6 +157,7 @@ dotnet_diagnostic.CA1835.severity = warning
 dotnet_diagnostic.CA1836.severity = warning
 dotnet_diagnostic.CA1837.severity = warning
 dotnet_diagnostic.CA1838.severity = none
+dotnet_diagnostic.CA1848.severity = none
 dotnet_diagnostic.CA1900.severity = warning
 dotnet_diagnostic.CA1901.severity = warning
 dotnet_diagnostic.CA1903.severity = warning

diff --git a/src/Directory.Build.props b/src/Directory.Build.props
@@ -1,13 +1,13 @@
 <Project>
   <PropertyGroup>
-    <TargetFramework>net7.0</TargetFramework>
+    <TargetFramework>net8.0</TargetFramework>
     <Copyright>Copyright © Workleap. $([System.DateTime]::UtcNow.ToString(yyyy))</Copyright>
     <Authors>Workleap</Authors>
     <Owners>Workleap</Owners>
     <PackageProjectUrl>https://github.com/gsoft-inc/wl-eventgrid-emulator</PackageProjectUrl>
     <PackageLicenseExpression>Apache-2.0</PackageLicenseExpression>
     <PublishRepositoryUrl>true</PublishRepositoryUrl>
-    <LangVersion>11</LangVersion>
+    <LangVersion>latest</LangVersion>
     <Nullable>enable</Nullable>
     <ImplicitUsings>enable</ImplicitUsings>
     <AnalysisLevel>latest-All</AnalysisLevel>

diff --git a/src/EventGridEmulator.Tests/CustomWebApplicationFactory.cs b/src/EventGridEmulator.Tests/CustomWebApplicationFactory.cs
diff --git a/src/EventGridEmulator.Tests/EmulatorValidationTests.cs b/src/EventGridEmulator.Tests/EmulatorValidationTests.cs
@@ -36,12 +36,12 @@ public async Task ValidatePublishAndSubscribeRoundTripForEventGridEvent()
 
         // Create and send an event to EventGrid
         var eventGridEvent = new EventGridEvent(
-            subject: "foo", 
-            eventType: "bar", 
-            dataVersion: "1.0", 
+            subject: "foo",
+            eventType: "bar",
+            dataVersion: "1.0",
             data: new DataModel(some: "data"));
         var response = await publisher.SendEventAsync(eventGridEvent);
-        
+
         // Assert that the message was successfully sent
         Assert.Equal(200, response.Status);
 
@@ -52,7 +52,7 @@ public async Task ValidatePublishAndSubscribeRoundTripForEventGridEvent()
         Assert.Equal("data", result?.Some);
         Assert.Equal($"{SubscriberConstants.DefaultTopicValue}{this.ExpectedTopic}", receivedTopic);
     }
-    
+
     [Fact]
     public async Task ValidatePublishAndSubscribeRoundTripForCloudEvent()
     {
@@ -73,7 +73,7 @@ public async Task ValidatePublishAndSubscribeRoundTripForCloudEvent()
         // Create and send an event to EventGrid
         var cloudEvent = new CloudEvent("foo", "bar", new DataModel(some: "data"));
         var response = await publisher.SendEventAsync(cloudEvent);
-        
+
         // Assert that the message was successfully sent
         Assert.Equal(200, response.Status);
 
@@ -85,7 +85,7 @@ public async Task ValidatePublishAndSubscribeRoundTripForCloudEvent()
         Assert.Equal($"{SubscriberConstants.DefaultTopicValue}{this.ExpectedTopic}", receivedTopic);
     }
 
-    private class DataModel
+    private sealed class DataModel
     {
         public DataModel(string some)
         {

diff --git a/src/EventGridEmulator.Tests/FactoryClientBuilder.cs b/src/EventGridEmulator.Tests/FactoryClientBuilder.cs
@@ -29,7 +29,7 @@ public HttpClient Build()
         return factory.CreateClient();
     }
 
-    private class Topic
+    private sealed class Topic
     {
         public Topic(string name, string url)
         {