Book Image

Mastering Hadoop

By : Karanth
Book Image

Mastering Hadoop

By: Karanth

Overview of this book

Do you want to broaden your Hadoop skill set and take your knowledge to the next level? Do you wish to enhance your knowledge of Hadoop to solve challenging data processing problems? Are your Hadoop jobs, Pig scripts, or Hive queries not working as fast as you intend? Are you looking to understand the benefits of upgrading Hadoop? If the answer is yes to any of these, this book is for you. It assumes novice-level familiarity with Hadoop.
Table of Contents (15 chapters)
14
Index

Summary


The number one goal of the streaming paradigm is to cater to low-latency applications. The Storm-on-YARN project has brought this paradigm to Hadoop. Stakeholders can now multiplex streaming and batch processing on a single Hadoop cluster and cater to different kinds of applications.

There are a number of streaming frameworks available, such as Microsoft SQL Server StreamInsight, S4, and Apache Storm, among others. Apache Storm is open source, part of the Apache Software Foundation, Hadoop integrated, and has a large community behind it, making it attractive for distributed stream processing.

Some key takeaways from this chapter are as follows:

  • The basic data model in Apache Storm is an unbounded sequence of tuples called Streams.

  • Long-standing queries are modeled as computational topologies. The data stream flows through these topologies.

  • Apache Storm provides the following primitives:

    • Spouts: They convert input data into streams

    • Bolts: They take an input stream, do some processing on...