Book Image

Mastering Hadoop

By : Karanth
Book Image

Mastering Hadoop

By: Karanth

Overview of this book

Do you want to broaden your Hadoop skill set and take your knowledge to the next level? Do you wish to enhance your knowledge of Hadoop to solve challenging data processing problems? Are your Hadoop jobs, Pig scripts, or Hive queries not working as fast as you intend? Are you looking to understand the benefits of upgrading Hadoop? If the answer is yes to any of these, this book is for you. It assumes novice-level familiarity with Hadoop.
Table of Contents (15 chapters)
14
Index

Storm on YARN

In Chapter 6, YARN – Bringing Other Paradigms to Hadoop, we built a YARN application that executes distributed shell commands. Storm is one such application that has been brought to YARN by Yahoo!. Any Hadoop cluster running YARN can now execute streaming workloads for low-latency real-time applications. The Application Master and client program to execute Storm are available for deployment. It is open sourced on GitHub at https://github.com/yahoo/storm-yarn.

Installing Apache Storm-on-YARN

Apache Storm-on-YARN can currently be installed from GitHub. This section assumes that the Hadoop 2.2.0 cluster is available.

Prerequisites

The following prerequisites are necessary to install Storm-on-YARN:

  • Java 7
  • Maven: This needs to be installed on the gateway machine to help compile and deploy the Storm-on-YARN Application Master and client:
    • wget http://mirror.symnds.com/software/Apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz
    • tar –zxvf apache-maven-3.1.1-bin...