Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

port conflict with random zookeeper ports #98

Open
dlipofsky opened this issue Oct 10, 2023 · 5 comments · May be fixed by #106
Open

port conflict with random zookeeper ports #98

dlipofsky opened this issue Oct 10, 2023 · 5 comments · May be fixed by #106

Comments

@dlipofsky
Copy link

About 1 out of every 50 test runs on my MacOS dev machine I get a port conflict on the randomly assigned zookeeper ports.
Would it be possible to validate that these ports are open before trying? Or catch and retry with different ports?
Or perhaps since zookeeper is not strictly needed with kafka 3.4 is it possible to use a config that does not use zookeeper?
I am currently doing

        kafka = provisionWith(defaultClusterConfig());
        kafka.start();

and getting

[INFO] Running com...KafkaMarketDataClientInteg1Test
2023-10-10 13:36:55 [Thread-8] ERROR o.a.c.test.TestingZooKeeperServer - From testing server (random state: false) for instance: InstanceSpec{dataDirectory=/var/folders/46/xk_p6xjx3w93pk8wv1tq_5wm0000gn/T/1696959413472-0, port=55521, electionPort=55522, quorumPort=55523, deleteDataDirectoryOnClose=true, serverId=
4, tickTime=-1, maxClientCnxns=-1, customProperties={}, hostname=127.0.0.1} org.apache.curator.test.InstanceSpec@59c48e5e
java.net.BindException: Address already in use
        at java.base/sun.nio.ch.Net.bind0(Native Method)
        at java.base/sun.nio.ch.Net.bind(Net.java:555)
        at java.base/sun.nio.ch.ServerSocketChannelImpl.netBind(ServerSocketChannelImpl.java:337)
        at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:294)
        at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:89)
        at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:81)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:676)
        at org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:109)
        at org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:105)
        at org.apache.curator.test.TestingZooKeeperMain.internalRunFromConfig(TestingZooKeeperMain.java:248)
        at org.apache.curator.test.TestingZooKeeperMain.runFromConfig(TestingZooKeeperMain.java:132)
        at org.apache.curator.test.TestingZooKeeperServer$1.run(TestingZooKeeperServer.java:158)
        at java.base/java.lang.Thread.run(Thread.java:833)
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 51.028 s <<< FAILURE! - in com...KafkaMarketDataClientInteg1Test
[ERROR] com...KafkaMarketDataClientInteg1Test.basicFlowWithFiltering  Time elapsed: 51.025 s  <<< ERROR!
java.lang.RuntimeException: Unable to start the embedded Kafka cluster.
        at net.mguenther.kafka.junit.EmbeddedKafkaCluster.start(EmbeddedKafkaCluster.java:66)
        at com...KafkaMarketDataClientInteg1Test.before(KafkaMarketDataClientInteg1Test.java:56)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
        Suppressed: java.lang.NullPointerException: Cannot invoke "java.util.Map.values()" because "this.brokers" is null
                at net.mguenther.kafka.junit.EmbeddedKafkaCluster.stop(EmbeddedKafkaCluster.java:77)
                at com...KafkaMarketDataClientInteg1Test.after(KafkaMarketDataClientInteg1Test.java:81)
                ... 3 more
Caused by: java.lang.RuntimeException: Unable to start an embedded ZooKeeper instance.
        at net.mguenther.kafka.junit.EmbeddedZooKeeper.start(EmbeddedZooKeeper.java:30)
        at net.mguenther.kafka.junit.EmbeddedKafkaCluster.start(EmbeddedKafkaCluster.java:45)
        ... 4 more
Caused by: org.apache.curator.test.FailedServerStartException: Timed out waiting for server startup
        at org.apache.curator.test.TestingZooKeeperMain.blockUntilStarted(TestingZooKeeperMain.java:147)
        at org.apache.curator.test.TestingZooKeeperServer.start(TestingZooKeeperServer.java:167)
        at org.apache.curator.test.TestingServer.<init>(TestingServer.java:117)
        at org.apache.curator.test.TestingServer.<init>(TestingServer.java:100)
        at org.apache.curator.test.TestingServer.<init>(TestingServer.java:63)
        at net.mguenther.kafka.junit.EmbeddedZooKeeper.start(EmbeddedZooKeeper.java:25)
        ... 5 more
@mguenther
Copy link
Owner

Haven't run into this issue yet. Out of curiosity: What are the specs of your system? Proper port management for the ZooKeeper testing server is a concern of the library that Kafka for JUnit uses. We're getting rid of it, once Kafka ships with a production-ready KRaft implementation that works out-of-the-box.

Kafka for JUnit is -in its current latest - tied to Apache Kafka 3.4.x, which comes with an early-access KRaft implementation. Hence, there's still the necessity for ZK. The earliest version that I'd feel comfortable with is 3.5.x, which ships with a production-ready opt-in for KRaft.

The easiest migration path for this library would be to wait for Apache Kafka 4.0.0, which will drop ZK entirely and enable KRaft out-of-the-box. However, I'll check if an opt-in for the next Kafka for JUnit version - which will target Apache Kafka 3.5.x - is feasible.

@dlipofsky
Copy link
Author

It's a MacBookPro circa 2019 running Ventura 13.5.2, 32 GB RAM.
lsof shows open listen ports are currently 29754, 49153-49163, 49172, 59578, 60010, 60012, 60014, 62290, 63342.
zookeeper ports from runs with a BindException are

port=49172, electionPort=49173, quorumPort=49174
port=50718, electionPort=50719, quorumPort=50720
port=50997, electionPort=50998, quorumPort=50999
port=55521, electionPort=55522, quorumPort=55523
port=57104, electionPort=57105, quorumPort=57106
port=61593, electionPort=61594, quorumPort=61595
port=62242, electionPort=62243, quorumPort=62244
port=65064, electionPort=65065, quorumPort=65066
port=65121, electionPort=65122, quorumPort=65123

Most of those ports don't show as currently in use, so I assume there were transient and no longer in use.

@dlipofsky
Copy link
Author

I would really appreciate a KRaft config with 3.5 if you can do it. Thank you.

@mguenther
Copy link
Owner

I took a look at it, but it is too much of a hassle to introduce an alternative provisioning mechanism, especially with the prospect of ZooKeeper being removed altogether in the very near future. Kafka for JUnit will remove ZK as soon as Kafka ships without it.

julianladisch added a commit to julianladisch/kafka-junit that referenced this issue Dec 17, 2024
@julianladisch
Ensure getUniqueEphemeralPorts returns only free ports
5acd3f6 
getUniqueEphemeralPorts doesn't check whether the ports are free or in use.

To avoid a port that is in use call new ServerSocket(0) that finds a
random ephemeral port that is free.

Fixes mguenther#98
@julianladisch julianladisch linked a pull request Dec 17, 2024 that will close this issue
Open
@julianladisch
Copy link
Contributor

org.testcontainers.containers.KafkaContainer supports KRaft mode (and kafka-native).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Ensure getUniqueEphemeralPorts returns only free ports julianladisch/kafka-junit
3 participants
@mguenther @julianladisch @dlipofsky