We have already talked a lot about Zookeeper in the previous sections. Zookeeper plays a very important role in Kafka architecture and it is very important for you to understand how it records the Kafka cluster state. Therefore, we are dedicating a separate section to the role of Zookeeper in the Kafka cluster. Kafka cannot work without Zookeeper. Kafka uses Zookeeper for the following functions:
- Choosing a controller: The controller is one of the brokers responsible for partition management with respect to leader election, topic creation, partition creation, and replica management. When a node or server shuts down, Kafka controllers elect partition leaders from followers. Kafka uses Zookeeper's metadata information to elect a controller. Zookeeper ensures that a new controller is elected in case the current controller crashes.
- Brokers metadata: Zookeeper...