Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
Storm Blueprints: Patterns for Distributed Real-time Computation
Mar, 2014 | 336 pages
No titles found
Table of Contents (17 chapters)
Storm Blueprints: Patterns for Distributed Real-time Computation
Storm Blueprints: Patterns for Distributed Real-time Computation
Credits
Credits
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Distributed Word Count
Distributed Word Count
Introducing elements of a Storm topology – streams, spouts, and bolts
Introducing the word count topology data flow
Implementing the word count topology
Introducing parallelism in Storm
Understanding stream groupings
Guaranteed processing
Summary
2
Configuring Storm Clusters
Configuring Storm Clusters
Introducing the anatomy of a Storm cluster
Introducing the Storm technology stack
Installing Storm on Linux
Submitting topologies to a Storm cluster
Automating the cluster configuration
A rapid introduction to Puppet
Summary
3
Trident Topologies and Sensor Data
Trident Topologies and Sensor Data
Examining our use case
Introducing Trident topologies
Introducing Trident spouts
Introducing Trident operations – filters and functions
Introducing Trident aggregators – Combiners and Reducers
Introducing the Trident state
Executing the topology
Summary
4
Real-time Trend Analysis
Real-time Trend Analysis
Use case
Architecture
Installing the required software
Introducing the sample application
Introducing the log analysis topology
The final topology
Running the log analysis topology
Summary
5
Real-time Graph Analysis
Real-time Graph Analysis
Use case
Architecture
A brief introduction to graph databases
Software installation
Setting up Titan to use the Cassandra storage backend
Graph data model
Connecting to the Twitter stream
Twitter graph topology
Implementing GraphState
Implementing GraphFactory
Implementing GraphTupleProcessor
Putting it all together – the TwitterGraphTopology class
Querying the graph with Gremlin
Summary
6
Artificial Intelligence
Artificial Intelligence
Designing for our use case
Establishing the architecture
Implementing the architecture
Summary
7
Integrating Druid for Financial Analytics
Integrating Druid for Financial Analytics
Use case
Integrating a non-transactional system
The topology
Implementing the architecture
Executing the implementation
Examining the analytics
Summary
8
Natural Language Processing
Natural Language Processing
Motivating a Lambda architecture
Examining our use case
Realizing a Lambda architecture
Designing the topology for our use case
Implementing the design
Examining the analytics
Batch processing / historical analysis
Hadoop
Summary
9
Deploying Storm on Hadoop for Advertising Analysis
Deploying Storm on Hadoop for Advertising Analysis
Examining the use case
Establishing the architecture
Configuring the infrastructure
Deploying the analytics
Performing the analytics
Deploying the topology
Executing the topology
Summary
10
Storm in the Cloud
Storm in the Cloud
Introducing Amazon Elastic Compute Cloud (EC2)
Introducing Apache Whirr
Configuring a Storm cluster with Whirr
Introducing Whirr Storm
Introducing Vagrant
Creating Storm-provisioning scripts
Summary
Index
Index

Introducing Amazon Elastic Compute Cloud (EC2)

Amazon EC2 is the central part of many remote compute services offered by Amazon. EC2 allows users to rent virtual compute resources hosted on Amazon's network infrastructure on demand.

We'll begin by setting up an EC2 account and manually launching a virtual machine on Amazon's EC2 infrastructure.

Setting up an AWS account

Establishing an AWS account is easy but requires an Amazon account. If you don't already have an Amazon account, sign up for one at http://www.amazon.com/.

With your Amazon account established, you can set up an AWS account at http://aws.amazon.com/.

The AWS Management Console

The AWS Management Console acts as the main administrative interface to all the cloud services that Amazon offers. We're primarily interested in the EC2 service, so let's begin by logging in to the EC2 Management Console as shown in the following screenshot:

Creating an SSH key pair

Before you can launch any EC2 instances, you will need a key pair. To create...

Previous Section
End of Section 2
Next Section