Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Storm Blueprints: Patterns for Distributed Real-time Computation
  • Table Of Contents Toc
Storm Blueprints: Patterns for Distributed Real-time Computation

Storm Blueprints: Patterns for Distributed Real-time Computation

By : Brian O'Neill
4.1 (8)
Buy this Book
close
close
Storm Blueprints: Patterns for Distributed Real-time Computation

Storm Blueprints: Patterns for Distributed Real-time Computation

4.1 (8)
By: Brian O'Neill
Buy this Book

Overview of this book

A blueprints book with 10 different projects built in 10 different chapters which demonstrate the various use cases of storm for both beginner and intermediate users, grounded in real-world example applications. Although the book focuses primarily on Java development with Storm, the patterns are more broadly applicable and the tips, techniques, and approaches described in the book apply to architects, developers, and operations. Additionally, the book should provoke and inspire applications of distributed computing to other industries and domains. Hadoop enthusiasts will also find this book a good introduction to Storm, providing a potential migration path from batch processing to the world of real-time analytics.
Table of Contents (17 chapters)
close
close
close
Storm Blueprints: Patterns for Distributed Real-time Computation
Icon
Storm Blueprints: Patterns for Distributed Real-time Computation
Credits
Icon
Credits
About the Authors
Icon
About the Authors
About the Reviewers
Icon
About the Reviewers
www.PacktPub.com
Icon
www.PacktPub.com
Preface
Icon
Preface
Lock Free Chapter
1
Distributed Word Count
Icon
Distributed Word Count
Icon
Introducing elements of a Storm topology – streams, spouts, and bolts
Icon
Introducing the word count topology data flow
Icon
Implementing the word count topology
Icon
Introducing parallelism in Storm
Icon
Understanding stream groupings
Icon
Guaranteed processing
Icon
Summary
2
Configuring Storm Clusters
Icon
Configuring Storm Clusters
Icon
Introducing the anatomy of a Storm cluster
Icon
Introducing the Storm technology stack
Icon
Installing Storm on Linux
Icon
Submitting topologies to a Storm cluster
Icon
Automating the cluster configuration
Icon
A rapid introduction to Puppet
Icon
Summary
3
Trident Topologies and Sensor Data
Icon
Trident Topologies and Sensor Data
Icon
Examining our use case
Icon
Introducing Trident topologies
Icon
Introducing Trident spouts
Icon
Introducing Trident operations – filters and functions
Icon
Introducing Trident aggregators – Combiners and Reducers
Icon
Introducing the Trident state
Icon
Executing the topology
Icon
Summary
4
Real-time Trend Analysis
Icon
Real-time Trend Analysis
Icon
Use case
Icon
Architecture
Icon
Installing the required software
Icon
Introducing the sample application
Icon
Introducing the log analysis topology
Icon
The final topology
Icon
Running the log analysis topology
Icon
Summary
5
Real-time Graph Analysis
chevron up
Icon
Real-time Graph Analysis
Icon
Use case
Icon
Architecture
Icon
A brief introduction to graph databases
Icon
Software installation
Icon
Setting up Titan to use the Cassandra storage backend
Icon
Graph data model
Icon
Connecting to the Twitter stream
Icon
Twitter graph topology
Icon
Implementing GraphState
Icon
Implementing GraphFactory
Icon
Implementing GraphTupleProcessor
Icon
Putting it all together – the TwitterGraphTopology class
Icon
Querying the graph with Gremlin
Icon
Summary
6
Artificial Intelligence
Icon
Artificial Intelligence
Icon
Designing for our use case
Icon
Establishing the architecture
Icon
Implementing the architecture
Icon
Summary
7
Integrating Druid for Financial Analytics
Icon
Integrating Druid for Financial Analytics
Icon
Use case
Icon
Integrating a non-transactional system
Icon
The topology
Icon
Implementing the architecture
Icon
Executing the implementation
Icon
Examining the analytics
Icon
Summary
8
Natural Language Processing
Icon
Natural Language Processing
Icon
Motivating a Lambda architecture
Icon
Examining our use case
Icon
Realizing a Lambda architecture
Icon
Designing the topology for our use case
Icon
Implementing the design
Icon
Examining the analytics
Icon
Batch processing / historical analysis
Icon
Hadoop
Icon
Summary
9
Deploying Storm on Hadoop for Advertising Analysis
Icon
Deploying Storm on Hadoop for Advertising Analysis
Icon
Examining the use case
Icon
Establishing the architecture
Icon
Configuring the infrastructure
Icon
Deploying the analytics
Icon
Performing the analytics
Icon
Deploying the topology
Icon
Executing the topology
Icon
Summary
10
Storm in the Cloud
Icon
Storm in the Cloud
Icon
Introducing Amazon Elastic Compute Cloud (EC2)
Icon
Introducing Apache Whirr
Icon
Configuring a Storm cluster with Whirr
Icon
Introducing Whirr Storm
Icon
Introducing Vagrant
Icon
Creating Storm-provisioning scripts
Icon
Summary
1
Index
Icon
Index

Architecture

The architecture for our application is relatively simple. We will create a Twitter client application that reads a subset of the Twitter firehose and writes each message to a Kafka queue as a JSON data structure. We'll then use the Kafka spout to feed that data into our storm topology. Finally, our storm topology will analyze the incoming messages and populate the graph database.

The Twitter client

Twitter provides a comprehensive RESTful API that in addition to a typical request-response interface also provides a streaming API that supports long-lived connections. The Twitter4J Java library (http://twitter4j.org/) offers full compatibility with the latest version of the Twitter API and takes care of all the low-level details (connection management, OAuth authentication, and JSON parsing) with a clean Java API. We will use Twitter4J to connect to the Twitter-streaming API.

Kafka spout

In the previous chapter, we developed a Logback Appender extension that allowed us to easily publish...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Storm Blueprints: Patterns for Distributed Real-time Computation
End of Section 5
Next Section
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful
Order Page Read Now

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon