Book Image

YARN Essentials

Book Image

YARN Essentials

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
YARN Essentials
Feb, 2015 | 176 pages
No titles found
Table of Contents (17 chapters)
YARN Essentials
YARN Essentials
Credits
Credits
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Need for YARN
Need for YARN
The redesign idea
Summary
2
YARN Architecture
YARN Architecture
Core components of YARN architecture
YARN scheduler policies
Recent developments in YARN architecture
Summary
3
YARN Installation
YARN Installation
Single-node installation
The fully-distributed mode
Operating Hadoop and YARN clusters
Web interfaces of the Ecosystem
Summary
4
YARN and Hadoop Ecosystems
YARN and Hadoop Ecosystems
The Hadoop 2 release
A short introduction to Hadoop 1.x and MRv1
MRv1 versus MRv2
Understanding where YARN fits into Hadoop
Old and new MapReduce APIs
Backward compatibility of MRv2 APIs
Practical examples of MRv1 and MRv2
Summary
5
YARN Administration
YARN Administration
Container allocation
Container configurations
YARN scheduling policies
YARN multitenancy application support
Administration of YARN
Summary
6
Developing and Running a Simple YARN Application
Developing and Running a Simple YARN Application
Running sample examples on YARN
Monitoring YARN applications with web GUI
YARN's MapReduce support
The YARN application workflow
Summary
7
YARN Frameworks
YARN Frameworks
Apache Samza
Storm-YARN
Apache Spark
Apache Tez
Apache Giraph
HOYA (HBase on YARN)
KOYA (Kafka on YARN)
Summary
8
Failures in YARN
Failures in YARN
ResourceManager failures
ApplicationMaster failures
NodeManager failures
Container failures
Hardware Failures
Summary
9
YARN – Alternative Solutions
YARN – Alternative Solutions
Mesos
Omega
Corona
Summary
10
YARN – Future and Support
YARN – Future and Support
What YARN means to the big data industry
Journey – present and future
YARN-supported frameworks
Summary
Index
Index

Chapter 4. YARN and Hadoop Ecosystems

This chapter discusses YARN with respect to Hadoop, since it is very important to know where exactly YARN fits in Hadoop 2 now.

Hadoop 2 has undergone a complete change in terms of architecture and components compared to Hadoop 1.

In this chapter, we will be cover the following topics:

  • A short introduction to Hadoop 1

  • The difference between MRv1 and MRv2

  • Where YARN fits in Hadoop 2

  • Old and new MapReduce APIs

  • Backward compatibility of MRv2 APIs

  • Practical examples of MRv1 and MRv2

Previous Section
End of Section 1
Next Section