Can I send kafka message header

[amitgurav04]

Can I send kafka message header along with data (or actual message).

Similar to sarama producer's ProducerMessage. https://github.com/Shopify/sarama/blob/master/async_producer.go#L166

[fullstacktester]

I would really like to see this feature as well. Kafka Headers have existed for over a year now but with no easy way to access them other via code. As a company we have Kafka deployed with Kafka Rest sitting on top. This was used mainly in non-production environments to publish/consume in a quick and easy way for testing. Since starting to use Kafka Headers though, we have no longer been able to do this.

Please could somebody take a look at this and let us know whether this is something that is likely to be (or hopefully already is) added to the backlog?

[nick-zh]

@ewencp any insight on this if this is planned in the near future?

[ewencp]

@nick-zh I'm not aware of a specific timeline for this feature in REST Proxy, but I'm not currently focused on REST Proxy (and haven't personally been for awhile now). Of course a community contribution would be welcome (we'd probably just want to do a bit of API design work early and up front to avoid too much refactoring/changes -- given multiple headers and the string key/binary value nature of them, we'd want to make sure the API would actually solve user use cases). In terms of Confluent's roadmap, we're aware of this request, but am not sure about exact timeline.

Out of curiosity, are your use cases primarily produce, consume, or both? If one is more critical, might reduce the scope to make it easier to get to.

[nick-zh]

Dear @ewencp
Thx very much for the feedback and the input, it is much appreciated.
I hope i can find time to look into this. Maybe as background, for us, it is in some cases technically not possible to have a steady producer connection, meaning we have handshake overhead for each message triggered, which i very much dislke. The proxy would currently solve that for us, but we really need headers.
In terms of priority, focus lies on the producer. So i will try to focus on that first. If it is easy to adapt for the consumer as well (which i hope it should), i will adapt it nevertheless.
If you receive any update regarding the timeline, an update would be much appreciated.
Have a great day!

[alozano3]

@ewencp I would be really interested on this feature. I would personally prefer the producer test case.
In fact I have been doing some tests on my own, sending custom http headers, then read these headers and build the kafka headers for the message.

I don't know if this is the correct way to do it but if it is , do you think it is worth to open a PR or do you plan to add this feature in short term?
Regards!

[nick-zh]

@alozano3 personally i think it would be nice to have it in record itself:

{
    "records": [
        {
            "key": "some key",
            "value": "some value"
            "headers": [
                {"Header": "HeaderValue"}
            ]
        }
    ]
}

What is your opinion on this @ewencp

[alozano3]

@nick-zh I think that's a good approach!

[pedrobacchini]

any news about this request?

[gaikwadgl20]

Hi,
Do we have any updates on this issue ?

[nick-zh]

I didn't pursue it any further, but from the comment of a linked pr it seems this has been implemented for v3 🎉