Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Overview of this book

Table of Contents (17 chapters)
Storm Blueprints: Patterns for Distributed Real-time Computation
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface
Index

Implementing the architecture


With the design in place, we can turn our attention to the implementation. The sequence diagram for the implementation is shown as follows:

The preceding diagram implements the state machine shown in the design. Once the real-time server is started, Druid polls the StormFirehose object using the hasMore() method. The contract with Druid specifies that the Firehose object's implementation should block until data is available. While Druid is polling and the Firehose object is blocking, Storm delivers tuples into the DruidState object's message buffer. When the batch is complete, Storm calls the commit() method on the DruidState object. At that point, the PartitionStatus is updated. The partition is put in progress and the implementation unblocks the StormFirehose object.

Druid begins pulling data from the StormFirehose object via the nextRow() method. When the StormFirehose object exhausts the contents of the partition, it places the partition in limbo, and releases...