Book Image

YARN Essentials

Book Image

YARN Essentials

Overview of this book

If you have a working knowledge of Hadoop 1.x but want to start afresh with YARN, this book is ideal for you. You will be able to install and administer a YARN cluster and also discover the configuration settings to fine-tune your cluster both in terms of performance and scalability. This book will help you develop, deploy, and run multiple applications/frameworks on the same shared YARN cluster.
Table of Contents (12 chapters)
Free Chapter
1
1. Need for YARN
9
9. YARN – Alternative Solutions
11
Index

Old and new MapReduce APIs


The new API (which is also known as Context Objects) was primarily designed to make the API easier to evolve in the future and is type incompatible with the old one.

The new API came into the picture from the 1.x release series. However, it was partially supported in this series. So, the old API is recommended for 1.x series:

Feature\Release

1.x

0.23

Old MapReduce API

Yes

Deprecated

New MapReduce API

Partial

Yes

MRv1 runtime (Classic)

Yes

No

MRv2 runtime (YARN)

No

Yes

The old and new API can be compared as follows:

Old API

New API

The old API is in the org.apache.hadoop.mapred package and is still present.

The new API is in the org.apache.hadoop.mapreduce package.

The old API used interfaces for Mapper and Reducer.

The new API uses Abstract Classes for Mapper and Reducer.

The old API used the JobConf, OutputCollector, and Reporter object to communicate with the MapReduce system.

The new API uses the context object to communicate with the MapReduce...