Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Overview of this book

Table of Contents (17 chapters)
Storm Blueprints: Patterns for Distributed Real-time Computation
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface
Index

Automating the cluster configuration


So far, we've configured a single-node pseudo-cluster manually from the command line. While this approach certainly works with small clusters, it will quickly become untenable as the cluster size increases. Consider the situation where one needs to configure clusters consisting of tens, hundreds, or even thousands of nodes. The configuration tasks can be automated using shell scripts, but even a shell script-based automation solution is questionable in terms of scalability.

Fortunately, there are a number of technologies available to help address the issue of configuration and provisioning of large numbers of managed servers. Both Chef and Puppet offer a declarative approach to configuration that allows you to define states (that is, what packages are installed and how they are configured) as well as classes of machines (for example, an Apache web server class machine needs to have the Apache httpd daemon installed).

Automating the process of provisioning...