Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Overview of this book

Table of Contents (17 chapters)
Storm Blueprints: Patterns for Distributed Real-time Computation
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface
Index

Integrating a non-transactional system


To extend on our previous example, we could develop a framework for the configuration that would allow the user to specify the dimensions along which they would like to aggregate events. Then, we could use that configuration in our topology to maintain a set of in-memory data sets to accumulate the aggregations, but any in-memory store is susceptible to faults. To address fault-tolerance, we could then make those aggregations persist in a database.

We would need to anticipate and support all the different types of aggregations the user would like to perform (for example, sum, average, geospatial, and so on). This seems like a substantial endeavor.

Fortunately, there are options for real-time analytics engines. One popular open-source option is Druid. The following article is taken from their whitepaper found at http://static.druid.io/docs/druid.pdf:

Druid is an open source, real-time analytical data store that supports fast ad-hoc queries on large-scale...