Book Overview & Buying
Table Of Contents

Apache Kafka 1.0 Cookbook

By : Estrada, Zinoviev

3 (2)

Buy this Book

Apache Kafka 1.0 Cookbook

3 (2)

By: Estrada, Zinoviev

Buy this Book

Overview of this book

Apache Kafka provides a unified, high-throughput, low-latency platform to handle real-time data feeds. This book will show you how to use Kafka efficiently, and contains practical solutions to the common problems that developers and administrators usually face while working with it. This practical guide contains easy-to-follow recipes to help you set up, configure, and use Apache Kafka in the best possible manner. You will use Apache Kafka Consumers and Producers to build effective real-time streaming applications. The book covers the recently released Kafka version 1.0, the Confluent Platform and Kafka Streams. The programming aspect covered in the book will teach you how to perform important tasks such as message validation, enrichment and composition.Recipes focusing on optimizing the performance of your Kafka cluster, and integrate Kafka with a variety of third-party tools such as Apache Hadoop, Apache Spark, and Elasticsearch will help ease your day to day collaboration with Kafka greatly. Finally, we cover tasks related to monitoring and securing your Apache Kafka cluster using tools such as Ganglia and Graphite. If you're looking to become the go-to person in your organization when it comes to working with Apache Kafka, this book is the only resource you need to have.

Preface

What this book covers

What you need for this book

Who this book is for

Sections

Conventions

Reader feedback

Customer support

Free Chapter

Configuring Kafka

Introduction

Installing Kafka

Running Kafka

Configuring Kafka brokers

Configuring Kafka topics

Creating a message console producer

Creating a message console consumer

Configuring the broker settings

Configuring threads and performance

Configuring the log settings

Configuring the replica settings

Configuring the ZooKeeper settings

Configuring other miscellaneous parameters

Kafka Clusters

Introduction

Configuring a single-node single-broker cluster – SNSB

SNSB – creating a topic, producer, and consumer

Configuring a single-node multiple-broker cluster – SNMB

SNMB – creating a topic, producer, and consumer

Configuring a multiple-node multiple-broker cluster – MNMB

Message Validation

Introduction

Modeling the events

Setting up the project

Reading from Kafka

Writing to Kafka

Running ProcessingApp

Coding the validator

Running the validator

Message Enrichment

Introduction

Geolocation extractor

Geolocation enricher

Currency price extractor

Currency price enricher

Running the currency price enricher

Modeling the events

Setting up the project

Open weather extractor

Location temperature enricher

Running the location temperature enricher

The Confluent Platform

Introduction

Installing the Confluent Platform

Using Kafka operations

Monitoring with the Confluent Control Center

Using the Schema Registry

Using the Kafka REST Proxy

Using Kafka Connect

Kafka Streams

Introduction

Setting up the project

Running the streaming application

Managing Kafka

Introduction

Managing consumer groups

Dumping log segments

Importing ZooKeeper offsets

Using the GetOffsetShell

Using the JMX tool

Using the MirrorMaker tool

Replaying log producer

Using state change log merger

Operating Kafka

Introduction

Adding or removing topics

Modifying message topics

Implementing a graceful shutdown

Balancing leadership

Expanding clusters

Increasing the replication factor

Decommissioning brokers

Checking the consumer position

Monitoring and Security

Introduction

Monitoring server statistics

Monitoring producer statistics

Monitoring consumer statistics

Connecting with the help of Graphite

Monitoring with the help of Ganglia

Implementing authentication using SSL

Implementing authentication using SASL/Kerberos

Third-Party Tool Integration

Introduction

Moving data between Kafka nodes with Flume

Writing to an HDFS cluster with Gobblin

Moving data from Kafka to Elastic with Logstash

Connecting Spark streams and Kafka

Ingesting data from Kafka to Storm

Pushing data from Kafka to Elastic

Inserting data from Kafka to SolrCloud

Building a Kafka producer with Akka

Building a Kafka consumer with Akka

Storing data in Cassandra

Running Kafka on Mesos

Reading Kafka with Apache Beam

Writing to Kafka from Apache Beam

Configuring Kafka

In this chapter, we will cover the following topics:

Installing Kafka
Running Kafka
Configuring Kafka brokers
Configuring Kafka topics
Creating a message console producer
Creating a message console consumer
Configuring the broker settings
Configuring threads and performance
Configuring the log settings
Configuring the replica settings
Configuring the Zookeeper settings
Configuring other miscellaneous parameters

Tech Concepts

Programming languages

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Your notes and bookmarks

Apache Kafka 1.0 Cookbook

By : Estrada, Zinoviev

Apache Kafka 1.0 Cookbook

By: Estrada, Zinoviev

Overview of this book

Confirmation

Buy this book with your credits?

Submit Your Feedback

Create a Free Account To Continue Reading

Sign in to activate your 7-day free access