Add example to make it easy to test brod_gssapi #6
Conversation
|
I created an issue with error logs and a detailed description of how to reproduce the problem with |
Will need to dig some more.... @kjellwinblad The problem is the version of Edit: Seems to be related to just pulling in via git deps, which suggests a build issue. If pull in brod via git in rebar.config at Edit: Hopefully this is my final edit :) Issue is the It'll work (4.0.2). Probably should do a PR to brod to bump to 4.0.2? Maybe also use hex deps vs git deps, though it would be good to have two profiles one that pulls in main/master from git and the other from hex (i.e, latest stable). |
|
Thank you @starbelly! I will make a PR to brod and see what I can do about the two profiles. |
Upgrading kafka_protocol to the latest version (4.0.3) fixes an issue with the brod_gssapi plugin: kafka4beam/brod_gssapi#6 (comment)
Upgrading kafka_protocol to the latest version (4.0.3) fixes an issue with the brod_gssapi plugin: kafka4beam/brod_gssapi#6 (comment)
|
@starbelly after fixing the version according to your suggestion 6ea28d0 (merged into the prod repo and new version of brod released) the brod client seem connects which is a big improvement. Unfortunately, I'm having trouble sending a message to a topic. I have created a topic mytest (this is done by the Which cause a crash with the following info: When the topic does not exists I get another error saying that the topic does not exists which indicate that the authentication is successful. Maybe the most interesting info from the error reason is @starbelly @zmstone do you have any idea what is happening here? It seems to me that the brod_producer is disconnecting when Kafka is expecting something else. What could be the reason for that? |
|
had a look together with @kjellwinblad |
|
As mentioned by @zmstone, the issue was fixed by changing the advitersed.listener config (6dae865). Apparently, this was an issue for brod but the not for Kafka's command line clients https://github.com/kafka4beam/brod_gssapi/blob/6dae865a5433cce6c63a2782134a3f922934020e/example/test_send_receive.sh. Thanks @zmstone! |
|
It's an issue for any client which needs to (re)connect the advertised endpoints. |
kjellwinblad
force-pushed
the
kjell/example
branch
from
6dae865 to
0d6e0ef
Compare
|
@starbelly @zmstone I will add commits to this PR to fix the github action that @zmstone added and also add a github action that will run the GSSAPI (Kerberos) Kafka setup to check that brod_gssapi can connect to a real system. So you do not need to fix the GitHub action in any other PR. |
I have added github actions for running with the latest version of brod, the master branch, and version 3.16.3. Now, it is also easy to test with a custom version of brod. See 27b4e60. |
Cool. I can remove the one I have in current PR I have open, or you can deal with conflicts. Let me know. As a side note to that PR. I'll push up docs tonight and then we can merge it. Wanted to get the tests cleaned up, but can do a follow up PR for that, rather not be blocking. |
I think you can keep it. I will deal with the conflicts. But it is up to you.
Yes, I will rebase and tidy up the commits after PR #5 has been merged. |
kjellwinblad
force-pushed
the
kjell/example
branch
2 times, most recently
from
ec6ef26 to
4b370d0
Compare
|
@starbelly I rebased on your latest changes and now it fails: https://github.com/kafka4beam/brod_gssapi/runs/6809888396?check_suite_focus=true . Do you have any idea about what the problem might be? Edit: Edit again: I think I use the wrong callback module in the settings. Will try to change that to see if it will work. |
|
@starbelly: I got it to work after chaning the callback module but I also think I found a bug. See
|
Indeed :) I'll fix that up, also why I like using maps when number params > 4 (as a guide line).
Maybe we can find some stats on this, the old version goes back to 2015 I believe? If that's right, it's quite old and I would think most have would upgraded since. |
It looks like the placement of the handshake version is still wrong but I can fix that in a commit to this PR. brod_gssapi/src/brod_gssapi.erl Line 43 in 5f23624 |
kjellwinblad
force-pushed
the
kjell/example
branch
from
477db0e to
90c00ce
Compare
Upgrading kafka_protocol to the latest version (4.0.3) fixes an issue with the brod_gssapi plugin: kafka4beam/brod_gssapi#6 (comment)
brod_gssapifails to connect to the Kafka instance in this example. This should be fixed before merge.This PR adds a
docker-composeexample that sets up Kerberos and Kafka (with SASL GSSAPI Kerberos authentication). The scriptexample/test_send_receive.shcan be used to test that the Kafka instance works once it has been started with theexample/upscript.The example also includes a small Erlang application (
example/brod_client) that usesbrod_gssapito send and receive a message to/from the Kafka instance. This can be started withdocker-compose up brod_client(from the example dir) once the Kafka instance has been started with theupscript. Unfortunately, this does not currently work. When looking at the Kafka logsdocker-compose logs kafkait seems like authentication is successful but that Kafka receives a package that it does not expect afterwards which causes it to close the connection. This could be related to the issue reported here kafka4beam/brod#383.