Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Book Image

Storm Blueprints: Patterns for Distributed Real-time Computation

Overview of this book

Table of Contents (17 chapters)
Storm Blueprints: Patterns for Distributed Real-time Computation
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface
Index

Introducing Trident aggregators – Combiners and Reducers


Akin to functions, aggregators allow topologies to combine tuples. Unlike functions, they replace tuple fields and values. There are three different types of aggregators: CombinerAggregator, ReducerAggregator, and Aggregator.

CombinerAggregator

A CombinerAggregator is used to combine a set of tuples into a single field. It has the following signature:

public interface CombinerAggregator {
   T init (TridentTuple tuple);
   T combine(T val1, T val2);
   T zero();
}

Storm calls the init() method with each tuple, and then repeatedly calls the combine() method until the partition is processed. The values passed into the combine() method are partial aggregations, the result of combining the values returned by calls to init(). Partitions are discussed more in the following sessions, but a partition is effectively a subset of a stream of tuples that resides on the same host. After combing the values from processing the tuples, Storm emits the...