Book Image

Learning Hadoop 2

By : Gerald Turkington, GABRIELE MODENA
Book Image

Learning Hadoop 2

By: Gerald Turkington, GABRIELE MODENA

Overview of this book

Table of Contents (18 chapters)
Learning Hadoop 2
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface
Index

Other computational frameworks


We've frequently discussed the myriad possibilities brought to the Hadoop platform by YARN. We went into details of two new models, Samza and Spark. Additionally, other more established frameworks such as Pig are also being ported to the framework.

To give a view of the much bigger picture in this section, we will illustrate the breadth of processing possible using YARN by presenting a set of computational models that are currently being ported to Hadoop on top of YARN.

Apache Storm

Storm (http://storm.apache.org) is a distributed computation framework written (mainly) in the Clojure programming language. It uses custom-created spouts and bolts to define information sources and manipulations to allow distributed processing of streaming data. A Storm application is designed as a topology of interfaces that creates a stream of transformations. It provides similar functionality to a MapReduce job with the exception that the topology will theoretically run indefinitely...