Book Image

Storm Real-time Processing Cookbook

By : Quinton Anderson
Book Image

Storm Real-time Processing Cookbook

By: Quinton Anderson

Overview of this book

<p>Storm is a free and open source distributed real-time computation system. Storm makes it easy to reliably process unbounded streams of data, doing for real-time processing what Hadoop did for batch processing. Storm is simple, can be used with any programming language, and is a lot of fun to use!<br />Storm Real Time Processing Cookbook will have basic to advanced recipes on Storm for real-time computation.<br /><br />The book begins with setting up the development environment and then teaches log stream processing. This will be followed by real-time payments workflow, distributed RPC, integrating it with other software such as Hadoop and Apache Camel, and more.</p>
Table of Contents (16 chapters)
Storm Real-time Processing Cookbook
Credits
About the Author
About the Reviewers
www.packtpub.com
Preface
Index

Deploying Storm on AWS using Pallet


The Storm deploy module is recommended by the community for the deployment of Storm clusters on AWS. It is available at https://github.com/nathanmarz/storm-deploy, and like Storm itself, was built by Nathan Marz. And like the Storm project, it has really excellent documentation. Because of this fact, this recipe is heavily based on the content of that wiki.

Storm deploy is based on Pallet. Pallet is a node provisioning, configuration and administration tool written in Clojure. It is designed to simplify small to midsize deployments. At this stage, it is useful to quickly introduce and position Pallet quickly within the context of infrastructure deployments, Storm, and the other chapters of this book.

Pallet has the following few properties that are worth mentioning:

  • It has no inherent dependencies that must be installed on the nodes being managed. This makes it highly portable, supporting just about any image out there.

  • Pallet has no central server. It can...