Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates for compatability with more recent Kafka #8

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
+115 −156
Open

Updates for compatability with more recent Kafka #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e38dc23
Updates for compatability with more recent Kafka
andpol Jul 23, 2019
519371e
Don't sort replicas, changes the partition leader
andpol Jul 24, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ Requires Java 1.7+

```
./kafka-assignment-generator.sh [options...] arguments...
--bootstrap_server VAL : Kafka Broker(s) to bootstrap
connection (comma-separated host:port
pairs)
--broker_hosts VAL : comma-separated list of broker
hostnames (instead of broker IDs)
--broker_hosts_to_remove VAL : comma-separated list of broker
Expand All @@ -40,41 +43,39 @@ Requires Java 1.7+
PRINT_CURRENT_BROKERS | ENT, PRINT_CURRENT_BROKERS,
PRINT_REASSIGNMENT] PRINT_REASSIGNMENT)
--topics VAL : comma-separated list of topics
--zk_string VAL : ZK quorum as comma-separated
host:port pairs
```

### Example: reassign partitions to all live hosts
```
./kafka-assignment-generator.sh --zk_string my-zk-host:2181 --mode PRINT_REASSIGNMENT
./kafka-assignment-generator.sh --bootstrap_server my-kafka-host:9092 --mode PRINT_REASSIGNMENT
```

The output JSON can then be fed into Kafka's reassign partitions command. See [here](http://kafka.apache.org/0100/ops.html#basic_ops_partitionassignment) for instructions.

### Example: reassign partitions to all but a few live hosts
This mode is useful for decommissioning or replacing a node. The partitions will be assigned to all live hosts, excluding the hosts that are specified.
```
./kafka-assignment-generator.sh --zk_string my-zk-host:2181 --mode PRINT_REASSIGNMENT --broker_hosts_to_remove misbehaving-host1,misbehaving-host2
./kafka-assignment-generator.sh --bootstrap_server my-kafka-host:9092 --mode PRINT_REASSIGNMENT --broker_hosts_to_remove misbehaving-host1,misbehaving-host2
```

The output JSON can then be fed into Kafka's reassign partitions command. See [here](http://kafka.apache.org/0100/ops.html#basic_ops_partitionassignment) for instructions.

### Example: reassign partitions to specific hosts
Note that in this mode, it is expected that every host that should own partitions should be specified, including existing ones.
```
./kafka-assignment-generator.sh --zk_string my-zk-host:2181 --mode PRINT_REASSIGNMENT --broker_hosts host1,host2,host3
./kafka-assignment-generator.sh --bootstrap_server my-kafka-host:9092 --mode PRINT_REASSIGNMENT --broker_hosts host1,host2,host3
```

The output JSON can then be fed into Kafka's reassign partitions command. See [here](http://kafka.apache.org/0100/ops.html#basic_ops_partitionassignment) for instructions.

### Example: print current brokers
```
./kafka-assignment-generator.sh --zk_string my-zk-host:2181 --mode PRINT_CURRENT_BROKERS
./kafka-assignment-generator.sh --bootstrap_server my-kafka-host:9092 --mode PRINT_CURRENT_BROKERS
```

### Example: print current assignment
```
./kafka-assignment-generator.sh --zk_string my-zk-host:2181 --mode PRINT_CURRENT_ASSIGNMENT
./kafka-assignment-generator.sh --bootstrap_server my-kafka-host:9092 --mode PRINT_CURRENT_ASSIGNMENT
```

# Building
Expand Down
11 changes: 3 additions & 8 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,10 @@
<artifactId>commons-lang3</artifactId>
<version>3.1</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>0.10.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.10.0.0</version>
<version>2.3.0</version>
</dependency>
</dependencies>

Expand Down Expand Up @@ -123,8 +118,8 @@
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<configuration>
<source>1.5</source>
<target>1.5</target>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
Expand Down
Loading