Book Image

YARN Essentials

By : Fasale, Nirmal Kumar
Book Image

YARN Essentials

By: Fasale, Nirmal Kumar

Overview of this book

If you have a working knowledge of Hadoop 1.x but want to start afresh with YARN, this book is ideal for you. You will be able to install and administer a YARN cluster and also discover the configuration settings to fine-tune your cluster both in terms of performance and scalability. This book will help you develop, deploy, and run multiple applications/frameworks on the same shared YARN cluster.
Table of Contents (12 chapters)
Free Chapter
1
1. Need for YARN
9
9. YARN – Alternative Solutions
11
Index

Operating Hadoop and YARN clusters

This is the final stage of Hadoop and YARN cluster setup and configuration. Here are the commands that need to be used to start and stop the Hadoop and YARN clusters.

Starting Hadoop and YARN clusters

To start Hadoop and the YARN cluster, use with the following procedure:

  1. Format a Hadoop distributed filesystem:
    $HADOOP_HOME/bin/hdfs namenode -format <cluster_name>
    
  2. The following command is used to start HDFS. Run it on the NameNode:
    $HADOOP_HOME/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR --script hdfs start namenode
    
  3. Run this command to start DataNodes on all slaves nodes:
    $HADOOP_HOME/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR --script hdfs start datanode
    
  4. Start YARN with the following command on the ResourceManager:
    $HADOOP_YARN_HOME/sbin/yarn-daemon.sh --config $HADOOP_CONF_DIR start resourcemanager
    
  5. Execute this command to start NodeManagers on all slaves:
    $HADOOP_YARN_HOME/sbin/yarn-daemon.sh --config $HADOOP_CONF_DIR start nodemanager
    
  6. Start a...