Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Overview of this book

Table of Contents (17 chapters)
Storm Blueprints: Patterns for Distributed Real-time Computation
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface
Index

Introducing the anatomy of a Storm cluster


Storm clusters follow a master/slave architecture similar to distributed computing technologies such as Hadoop but with slightly different semantics. In a master/slave architecture, there is typically a master node that is either statically assigned through configuration or dynamically elected at runtime. Storm uses the former approach. While the master/slave architecture can be criticized as a setup that introduces a single point of failure, we'll show that Storm is semi-tolerant of a master node failure.

A Storm cluster consists of one master node (called nimbus) and one or more worker nodes (called supervisors). In addition to the nimbus and supervisor nodes, Storm also requires an instance of Apache ZooKeeper, which itself may consist of one or more nodes as shown in the following diagram:

Both the nimbus and supervisor processes are daemon processes provided by Storm and do not need to be isolated from individual machines. In fact, it is possible...