Book Image

Learning Storm

By : Ankit Jain, Anand Nalya
Book Image

Learning Storm

By: Ankit Jain, Anand Nalya

Overview of this book

<p>Starting with the very basics of Storm, you will learn how to set up Storm on a single machine and move on to deploying Storm on your cluster. You will understand how Kafka can be integrated with Storm using the Kafka spout.</p> <p>You will then proceed to explore the Trident abstraction tool with Storm to perform stateful stream processing, guaranteeing single message processing in every topology. You will move ahead to learn how to integrate Hadoop with Storm. Next, you will learn how to integrate Storm with other well-known Big Data technologies such as HBase, Redis, and Kafka to realize the full potential of Storm.</p> <p>Finally, you will perform in-depth case studies on Apache log processing and machine learning with a focus on Storm, and through these case studies, you will discover Storm's realm of possibilities.</p>

Related Content you might be interested in

Current Title:

Small book image
Learning Storm
Ankit Jain | Anand Nalya
Aug, 2014 | 252 pages
No titles found
Table of Contents (16 chapters)
Learning Storm
Learning Storm
Credits
Credits
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Setting Up Storm on a Single Machine
Setting Up Storm on a Single Machine
Features of Storm
Storm components
The Storm data model
Summary
2
Setting Up a Storm Cluster
Setting Up a Storm Cluster
Setting up a distributed Storm cluster
Deploying a topology on a remote Storm cluster
Configuring the parallelism of a topology
Rebalancing the parallelism of a topology
Stream grouping
Guaranteed message processing
Summary
3
Monitoring the Storm Cluster
Monitoring the Storm Cluster
Starting to use the Storm UI
Monitoring a topology using the Storm UI
Cluster statistics using the Nimbus thrift client
Summary
4
Storm and Kafka Integration
Storm and Kafka Integration
The Kafka architecture
Setting up Kafka
A sample Kafka producer
Integrating Kafka with Storm
Summary
5
Exploring High-level Abstraction in Storm with Trident
Exploring High-level Abstraction in Storm with Trident
Introducing Trident
Understanding Trident's data model
Writing Trident functions, filters, and projections
Trident repartitioning operations
Trident aggregators
Utilizing the groupBy operation
A non-transactional topology
A sample Trident topology
Maintaining the topology state with Trident
A transactional topology
The opaque transactional topology
Distributed RPC
When to use Trident
Summary
6
Integration of Storm with Batch Processing Tools
Integration of Storm with Batch Processing Tools
Exploring Apache Hadoop
Installing Apache Hadoop
Integration of Storm with Hadoop
Deploying Storm-Starter topologies on Storm-YARN
Summary
7
Integrating Storm with JMX, Ganglia, HBase, and Redis
Integrating Storm with JMX, Ganglia, HBase, and Redis
Monitoring the Storm cluster using JMX
Monitoring the Storm cluster using Ganglia
Integrating Storm with HBase
Integrating Storm with Redis
Summary
8
Log Processing with Storm
Log Processing with Storm
Server log-processing elements
Producing the Apache log in Kafka
Splitting the server log line
Identifying the country, the operating system type, and the browser type from the logfile
Extracting the searched keyword
Persisting the process data
Defining a topology and the Kafka spout
Deploying a topology
MySQL queries
Summary
9
Machine Learning
Machine Learning
Exploring machine learning
Using Trident-ML
The use case – clustering synthetic control data
Producing a training dataset into Kafka
Building a Trident topology to build the clustering model
Summary
Index
Index

About the Authors

Ankit Jain holds a Bachelor's degree in Computer Science Engineering. He has 4 years of experience in designing and architecting solutions for the Big Data domain and has been involved with several complex engagements. His technical strengths include Hadoop, Storm, S4, HBase, Hive, Sqoop, Flume, ElasticSearch, Machine Learning, Kafka, Spring, Java, and J2EE. He is currently employed with Impetus Infotech Pvt. Ltd.

He also shares his thoughts on his personal blog at http://ankitasblogger.blogspot.in/. You can follow him on Twitter at @mynameisanky. He spends most of his time reading books and playing with different technologies. When not at work, he spends time with his family and friends watching movies and playing games.

I would like to thank my family and colleagues for always being there for me. Special thanks to the Packt Publishing team; without you guys, this work would not have been possible.

Anand Nalya is a full stack engineer with over 8 years of extensive experience in designing, developing, deploying, and benchmarking Big Data and web-scale applications for both start-ups and enterprises. He focuses on reducing the complexity in getting things done with brevity in code.

He blogs about Big Data, web applications, and technology in general at http://anandnalya.com/. You can also follow him on Twitter at @anandnalya. When not working on projects, he can be found stargazing or reading.

I would like to thank my wife, Nidhi, for putting up with so many of my side projects and my family members who are always there for me. Special thanks to my colleagues who helped me validate the writing, and finally, the reviewers and editors at Packt Publishing, without whom this work would not have been possible.

Previous Section
Next Chapter