Can we enable SASL/SCRAM on Windows without GSSAPI?

[Ilmarii]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

SASL is currently disabled due to build issues with krb5-src on WIndows (#3081 (comment))

Attempted Solutions

But if we only need SCRAM mechanisms krb5-src is not needed, sasl2-sys now has a separate feature for this: https://github.com/MaterializeInc/rust-sasl/blob/master/sasl2-sys/Cargo.toml#L38
I tried to build with this feature and it work fine for me.

Proposal

I suppose changes to rust-rdkafka will be required.
I added these features for my build:
rust-rdkafka: scram-vendored = ["rdkafka-sys/scram-vendored"]
rdkafka-sys: scram-vendored = ["ssl", "sasl2-sys/scram", "sasl2-sys/openssl-vendored"]
And for vector added "rdkafka?/scram-vendored" to default-msvc feature.

References

#4866

Version

0.28.1

[Neustradamus]

Linked to:

• State of Play scram-sasl/info#1

[sproberts92]

I would be keen for this to be enabled. Is there more to it than just adding the features described above?

[sproberts92]

I have opened a PR to rust-rdkafka to add those features that you suggested @Ilmarii.

[Neustradamus]

Thanks a lot @sproberts92!

Your PR is here:

• scram-vendored feature fede1024/rust-rdkafka#528

[sproberts92]

Hmm following on from @benesch's statement on the rust-rdkafka PR that we probably don't need a change to rust-rdkafka, I went back and did some testing with Vector on Windows.

Personally, I first noticed the lack of SCRAM support in Vector on Windows in 0.23.0, and this issue was opened on Oct 18, which would have been 0.24.2. I'd be willing to bet that none of us have really tested it again since, and assumed there was still no support... so it was much to my surprise when I tested 0.26.0 and I was able to use SASL/SCRAM successfully.

I can confirm that 0.25.2 still shows the familiar

2022-12-24T12:36:50.330949Z ERROR vector::topology: Configuration error. error=Sink "kafka": creating kafka producer failed: Client creation error: No provider for SASL mechanism SCRAM-SHA-256: recompile librdkafka with openssl support. Current build options: PLAIN WindowsSSPI(GSSAPI)

whereas 0.26.0 is able to produce without any issue.

Tested using the config below with Vector installed in a fresh Windows 10 VM, and an appropriately configured Kafka cluster:

[sinks.kafka] 
type = "kafka" 
inputs = ["parse_logs"] 
encoding.codec = "json" 
bootstrap_servers = "foo.bar:9093" 
sasl.enabled = true 
sasl.mechanism = "SCRAM-SHA-256" 
sasl.username = "test" 
sasl.password = "test" 
topic = "test"

I've had a bit of a look at the diff between 0.25.2 and 0.26.0 but as I'm less intimately familiar with the Cargo.toml of this project it's not obvious to me what the key difference is.

@Ilmarii are you able to confirm the same, that 0.26.0 now works with SASL/SCRAM on Windows? I've only confirmed at home for now and should also be able to test at the office later next week.

[jszwedko]

Interesting, thanks for digging into that @sproberts92 . I'm also not seeing anything jump out in the Cargo.toml diff. The upgrade of rdkafka/librdkafka seems like the most suspicious but looking through the changelogs there I'm not seeing anything either. We've already been vendoring openssl. I'll give @Ilmarii a chance to confirm, but otherwise it looks like we can close this out.

[jszwedko]

Closing this out since it seems to be complete.