Book Image

Building Data Streaming Applications with Apache Kafka

By : Chanchal Singh, Manish Kumar
Book Image

Building Data Streaming Applications with Apache Kafka

By: Chanchal Singh, Manish Kumar

Overview of this book

Apache Kafka is a popular distributed streaming platform that acts as a messaging queue or an enterprise messaging system. It lets you publish and subscribe to a stream of records, and process them in a fault-tolerant way as they occur. This book is a comprehensive guide to designing and architecting enterprise-grade streaming applications using Apache Kafka and other big data tools. It includes best practices for building such applications, and tackles some common challenges such as how to use Kafka efficiently and handle high data volumes with ease. This book first takes you through understanding the type messaging system and then provides a thorough introduction to Apache Kafka and its internal details. The second part of the book takes you through designing streaming application using various frameworks and tools such as Apache Spark, Apache Storm, and more. Once you grasp the basics, we will take you through more advanced concepts in Apache Kafka such as capacity planning and security. By the end of this book, you will have all the information you need to be comfortable with using Apache Kafka, and to design efficient streaming data applications with it.
Table of Contents (14 chapters)

Understanding Zookeeper authentication

Zookeeper is the metadata service for Kafka. SASL-enabled Zookeeper services first authenticate access to metadata stored in Zookeeper. Kafka brokers need to authenticate themselves using Kerberos to use Zookeeper services. If valid, the Kerberos ticket is presented to Zookeeper, it then provides access to the metadata stored in it. After valid authentication, Zookeeper establishes connecting user or service identity. This identity is then used to authorize access to metadata Znodes guarded by ACLs.

One important thing for you to understand is that Zookeeper ACLs restrict modifications to Znodes. Znodes can be read by any client. The philosophy behind this behavior is that sensitive data is not stored in Zookeeper. However, modifications by an unauthorized user can disrupt your cluster's behavior. Hence, Znodes are world readable, but...