Increasing the replication factor

In cases where more machines are added to the Kafka cluster, increasing the replication factor means moving replicas for a topic to these new machines.

Getting ready

For this recipe, Kafka must be installed, ZooKeeper running, and the broker running with some topics created on it with some replicas. Start new nodes and add them to this cluster.

How to do it...

This example increases the replication factor of partition 0 of the topic topic_1 from 2 to 4. Before the increment, the partition's only replica existed on brokers 3 and 4. This example adds more replicas on brokers 5 and 6.

1. Create a JSON file named increase-replication.json with this code:

$cat increase-replication.json{"version":1,"partitions":[{"topic":"topic_1","partition":0,"replicas":[3,4,5,6]}]}

2. Then, run the following command:

$ bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file increase-replication-factor.json --execute

How it works...

At the beginning, topic_1 was...