Decommissioning brokers

As Kafka clusters can be expanded, they can also be shortened. There are cases where it is necessary to remove some nodes. Removing some Kafka nodes from a cluster is called decommissioning. Decommissioning is not automatic; some reassignment must be applied to allow replicas to move to the live brokers.

Getting ready

For this recipe, Kafka must be installed, ZooKeeper running, and a Kafka cluster running with at least three nodes. A topic called topic_1 with replication factor 3 should be running on the cluster.

How to do it...

1. First, gracefully shut down the broker to be removed
2. Once it is shut down, create a JSON file named change-replication.json with the following content:

{"version":1, "partitions":[{"topic":"topic_1","partition":0,"replicas":[1,2]}]}  

3. Reassign the topic to the two living brokers with the reassign-partitions command:

$ bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file change-replication.json --execute

How it works...