With expanding the Kafka cluster also comes the scenario under which you might have to remove some nodes. The decommissioning of brokers is not automatic and you need to generate and apply the reassignment settings so that the replicas are moved to the other remaining brokers.
You should have a Kafka cluster up-and-running with at least three nodes. You can now create a topic with a replication factor of 3.
You can gracefully shutdown one of the broker nodes that you want to decommission. Once that broker node has shut down gracefully, perform the following steps:
Create a JSON file named
change-replication-factor.jsonwith the following code:{"version":1, "partitions":[{"topic":"mytesttopic","partition":0,"replicas":[1,2]}]}Run the following command:
> bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file change-replication-factor.json --execute