Sign In Start Free Trial

Book Overview & Buying
Table Of Contents
Feedback & Rating

YARN Essentials

By : Fasale, Nirmal Kumar

4.2 (5)

YARN Essentials

4.2 (5)

By: Fasale, Nirmal Kumar

Overview of this book

If you have a working knowledge of Hadoop 1.x but want to start afresh with YARN, this book is ideal for you. You will be able to install and administer a YARN cluster and also discover the configuration settings to fine-tune your cluster both in terms of performance and scalability. This book will help you develop, deploy, and run multiple applications/frameworks on the same shared YARN cluster.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Free Chapter

1. Need for YARN

1. Need for YARN

The redesign idea

Summary

2. YARN Architecture

2. YARN Architecture

Core components of YARN architecture

YARN scheduler policies

Recent developments in YARN architecture

Summary

3. YARN Installation

3. YARN Installation

Single-node installation

The fully-distributed mode

Operating Hadoop and YARN clusters

Web interfaces of the Ecosystem

Summary

4. YARN and Hadoop Ecosystems

4. YARN and Hadoop Ecosystems

The Hadoop 2 release

A short introduction to Hadoop 1.x and MRv1

MRv1 versus MRv2

Understanding where YARN fits into Hadoop

Old and new MapReduce APIs

Backward compatibility of MRv2 APIs

Practical examples of MRv1 and MRv2

Summary

5. YARN Administration

5. YARN Administration

Container allocation

Container configurations

YARN scheduling policies

YARN multitenancy application support

Administration of YARN

Summary

6. Developing and Running a Simple YARN Application

6. Developing and Running a Simple YARN Application

Running sample examples on YARN

Monitoring YARN applications with web GUI

YARN's MapReduce support

The YARN application workflow

Summary

7. YARN Frameworks

7. YARN Frameworks

Apache Samza

Storm-YARN

Apache Spark

Apache Tez

Apache Giraph

HOYA (HBase on YARN)

KOYA (Kafka on YARN)

Summary

8. Failures in YARN

8. Failures in YARN

ResourceManager failures

ApplicationMaster failures

NodeManager failures

Container failures

Hardware Failures

Summary

9. YARN – Alternative Solutions

9. YARN – Alternative Solutions

Mesos

Omega

Corona

Summary

10. YARN – Future and Support

10. YARN – Future and Support

What YARN means to the big data industry

Journey – present and future

YARN-supported frameworks

Summary

Index

Index

Customer Reviews

4.2 (5)

5 star

20%

4 star

80%

3 star

0%

2 star

0%

1 star

0%

Chapter 6. Developing and Running a Simple YARN Application

In the previous chapters, we discussed the concepts of the YARN architecture, cluster setup, and administration. Now in this chapter, we will focus more on MapReduce applications with YARN and its ecosystems, with some hands-on examples. You previously learned about when a client submits an application request to the YARN cluster and how YARN registers the application, allocates the required containers for its execution, and monitors the application while it's running. Now, we will see some practical use cases of YARN.

In this chapter, we will discuss:

Running sample applications on YARN
Developing YARN examples
Application monitoring and tracking

Now, let's start by running some of the sample applications that come as a part of the YARN distribution bundle.

Visually different images

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

YARN Essentials

Search

Your notes and bookmarks