3. Understanding Storm Internals by Examples | Learning Real-time Analytics with Storm and Cassandra

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents

Learning Real-time Analytics with Storm and Cassandra

By : Shilpi Saxena

3.5 (2)

Learning Real-time Analytics with Storm and Cassandra

3.5 (2)

By: Shilpi Saxena

Overview of this book

If you want to efficiently use Storm and Cassandra together and excel at developing production-grade, distributed real-time applications, then this book is for you. No prior knowledge of using Storm and Cassandra together is necessary. However, a background in Java is expected.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Free Chapter

1. Let's Understand Storm

1. Let's Understand Storm

Distributed computing problems

Solutions for complex distributed use cases

A high-level view of various components of Storm

Delving into the internals of Storm

Quiz time

Summary

2. Getting Started with Your First Topology

2. Getting Started with Your First Topology

Prerequisites for setting up Storm

Components of a Storm topology

Executing a sample Storm topology – local mode

Executing the topology in the distributed mode

Executing the topology from Command Prompt

Quiz time

Summary

3. Understanding Storm Internals by Examples

3. Understanding Storm Internals by Examples

Customizing Storm spouts

Anchoring and acking

Stream groupings

Quiz time

Summary

4. Storm in a Clustered Mode

4. Storm in a Clustered Mode

The Storm cluster setup

Zookeeper configurations

Storm configurations

Storm monitoring tools

Quiz time

Summary

5. Storm High Availability and Failover

5. Storm High Availability and Failover

An overview of RabbitMQ

Installing the RabbitMQ cluster

Integrating Storm with RabbitMQ

Building high availability of components

The Storm isolation scheduler

Quiz time

Summary

6. Adding NoSQL Persistence to Storm

6. Adding NoSQL Persistence to Storm

The advantages of Cassandra

Columnar database fundamentals

Setting up the Cassandra cluster

Multiple data centers

Introduction to CQLSH

Introduction to CLI

Using different client APIs to access Cassandra

Storm topology wired to the Cassandra store

The best practices for Storm/Cassandra applications

Quiz time

Summary

7. Cassandra Partitioning, High Availability, and Consistency

7. Cassandra Partitioning, High Availability, and Consistency

Consistent hashing

Replication in Cassandra and strategies

Cassandra consistency

Quiz time

Summary

8. Cassandra Management and Maintenance

8. Cassandra Management and Maintenance

Cassandra – gossip protocol

Cassandra cluster scaling – adding a new node

Cassandra cluster – replacing a dead node

The replication factor

The nodetool commands

Cassandra fault tolerance

Cassandra monitoring systems

Quiz time

Summary

9. Storm Management and Maintenance

9. Storm Management and Maintenance

Scaling the Storm cluster – adding new supervisor nodes

Scaling the Storm cluster and rebalancing the topology

Setting up workers and parallelism to enhance processing

Storm troubleshooting

Quiz time

Summary

10. Advance Concepts in Storm

10. Advance Concepts in Storm

Building a Trident topology

Understanding the Trident API

Examples and illustrations

Quiz time

Summary

11. Distributed Cache and CEP with Storm

11. Distributed Cache and CEP with Storm

The need for distributed caching in Storm

Introduction to memcached

Introduction to the complex event processing engine

Quiz time

Summary

A. Quiz Answers

Chapter 1

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Chapter 7

Chapter 8

Chapter 9

Chapter 10

Chapter 11

Index

Index

Customizing Storm spouts

You have explored and understood WordCount topology provided by the Storm-starter project in previous chapters. Now it's time we move on to the next step, the do it yourself journey with Storm; so let's take up the next leap and do some exciting stuff with our own spouts that read from various sources.

Creating FileSpout

Here we will create our own spout to read the events or tuples from a file source and emit them into the topology; we would substitute spout in place of RandomSentenceSpout we used in the WordCount topology in the previous chapter.

To start, copy the project we created in Chapter 2, Getting Started with Your First Topology, into a new project and make the following changes in RandomSentenceSpout to make a new class called FileSpout within the Storm-starter project.

Now we will make changes in FileSpout so that it reads sentences from a file as shown in the following code:

public class FileSpout extends BaseRichSpout {
  //declaration section
  SpoutOutputCollector...

Visually different images

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Learning Real-time Analytics with Storm and Cassandra

Search

Your notes and bookmarks