Balancing leadership

A leader broker of a topic partition can be crashed or stopped, and then the leadership is transferred to another replica. This might produce an imbalance in the lead Kafka brokers (an imbalance is when the leader is dead or unreachable). To recover from this imbalance, we need balancing leadership.

Getting ready

For this recipe, a Kafka cluster setup with several nodes is needed. One of the Kafka nodes is down, and subsequently, it is restored.

How to do it...

Run the following command from the Kafka installation directory:

$ bin/kafka-preferred-replica-election.sh --zookeeperlocalhost:2181/chroot  

How it works...

If the list of replicas for a partition is [3, 5, 8], then node 3 is preferred as the leader, rather than nodes 5 or 8. This is because it is earlier in the replica list. By running this command, we tell the Kafka cluster to try to restore leadership to the restored replicas.

To explain how it works, suppose that after the leader stops, new Kafka nodes join the cluster...