Book Image

Mastering Hadoop

By : Karanth
Book Image

Mastering Hadoop

By: Karanth

Overview of this book

Do you want to broaden your Hadoop skill set and take your knowledge to the next level? Do you wish to enhance your knowledge of Hadoop to solve challenging data processing problems? Are your Hadoop jobs, Pig scripts, or Hive queries not working as fast as you intend? Are you looking to understand the benefits of upgrading Hadoop? If the answer is yes to any of these, this book is for you. It assumes novice-level familiarity with Hadoop.
Table of Contents (15 chapters)
14
Index

The YARN architecture


The following figure illustrates the architecture of a YARN-based cluster. There are five major component types in a YARN cluster. They are as follows:

  • Resource Manager (RM): This is a per-cluster daemon that is solely responsible for allocating and managing resources available within the cluster.

  • Node Manager (NM): This is a per-node daemon that is responsible for local resource management. It is also the node-local representative of the RM.

  • Application Master (AM): This is a per-application daemon that encapsulates all application-specific logic and libraries. The AM is responsible for negotiating resources from the RM and working with the NM to execute them to completion.

  • Container: This is an abstract representation of a resource set that is given to a particular application. The AM is a specialized container that is used to bootstrap and manage the entire application's life cycle.

  • Client: This is the entity in the cluster that can submit applications to the RM...