AWS Glue Schema decoding

[davido912]

Hey hey!
Was wondering, is the AWS Glue Schema supported for decoding and encoding events?

I see here: https://www.benthos.dev/docs/components/processors/schema_registry_decode that only Confluent is mentioned.

Thanks!

[mihaitodor]

I believe this will require an enhancement to the schema_registry_* processors, since it looks like AWS Glue Schema uses a different approach: https://lenses.io/blog/2023/04/aws_glue_schema_registry_that_binds_apache_kafka/

AWS Glue Schema Registry uses a payload approach to identify schemas, utilizing 18 bytes. The schema identification process begins with the first byte (Byte 0), which is a version number. The following Byte 1 denotes whether compression is being used or not, and Bytes 2-17 hold a 128-bit UUID, representing the schema version ID.

The deserialization process considers the first Byte value, and if it is set to 3, then the Glue-based serialization is used. Otherwise, a fallback deserializer is employed. In cases where the compression byte is set, its value must be 5. This ensures that the Avro content is compressed using Zlib.

[davido912]

Answers my question :D
Probably could be a nice feature to have on the roadmap