Service Bus: Bulk publish does not apply metadata properties to events

[bely300]

Expected Behavior

When publishing events to Azure Service Bus Topic via the Dapr BulkPublishEventAsync method, the metadata properties and values should be applied to all events in the batch of events.

Actual Behavior

None of the metadata properties and values are applied to the events in the call to the BulkPublishEventAsync method.

Steps to Reproduce the Problem

using System;
using System.Collections.Generic;
using Dapr.Client;

const string PubsubName = "my-pubsub-name";
const string TopicName = "topic-a";
IReadOnlyList<object> BulkPublishData = new List<object>() {
    new { Id = "17", Amount = 10m },
    new { Id = "18", Amount = 20m },
    new { Id = "19", Amount = 30m }
};

using var client = new DaprClientBuilder().Build();

var metadata = new Dictionary<string, string>() { { "PartitionId", "RandomIDValue" } };
var res = await client.BulkPublishEventAsync(PubsubName, TopicName, BulkPublishData, metadata);

Reading events back via Dapr or viewing them in the Service Bus topic/Subscription in the Azure Portal, the events will not contain the expected metadata properties.

The issue is related to a mis-match between the way the Dapr .Net SDK handles the metadata versus how the Dapr runtime processes the passed in metadata.

The Dapr .Net SDK only packages up a single metadata map in the BulkPublishRequest object, so expects the Dapr runtime will apply the metadata to each event before writing to the Azure Service Bus Topic. See DaprClientGrpc.MakeBulkPublishRequest(....)

The logic on the runtime side, implemented in the 'components-contrib' repo ignores the metadata in the bulkPublishRequest object. Instead the logic walks through each event and processes any metadata object associated with the individual event. Refer to the 'PublishPubSubBulk' method in 'components-contrib/common/component/azure/servicebus/publisher.go'.

Release Note

RELEASE NOTE:

[WhitWaldo]

I'll see if I can't squeeze this into the 1.15 release. Thanks for spotting this @bely300 !

[cwalsh-truelearn]

Is this going to be included in 1.16? For our use case it causes event filtering on the metadata properties to not work correctly for the bulk published events, so single publish must be used. Not blocking anything but would be nice to utilize the bulk operations for these cases.

[WhitWaldo]

Is this going to be included in 1.16? For our use case it causes event filtering on the metadata properties to not work correctly for the bulk published events, so single publish must be used. Not blocking anything but would be nice to utilize the bulk operations for these cases.

@cwalsh-truelearn That's the goal, but contributions to ensure it gets there are always welcome to help ensure that. The goal is to release 1.15 imminently and then start working on 1.16 (anticipating a release in April-May).