Book Image

YARN Essentials

By : Fasale, Nirmal Kumar
Book Image

YARN Essentials

By: Fasale, Nirmal Kumar

Overview of this book

If you have a working knowledge of Hadoop 1.x but want to start afresh with YARN, this book is ideal for you. You will be able to install and administer a YARN cluster and also discover the configuration settings to fine-tune your cluster both in terms of performance and scalability. This book will help you develop, deploy, and run multiple applications/frameworks on the same shared YARN cluster.
Table of Contents (12 chapters)
Free Chapter
1
1. Need for YARN
9
9. YARN – Alternative Solutions
11
Index

Corona


Corona is another work from Facebook, which is now open-sourced and hosted on the GitHub repository at https://github.com/facebookarchive/hadoop-20/tree/master/src/contrib/corona.

Facebook, with its huge peta-scale quantity of data, suffered serious performance-related issues with the classic MapReduce framework because of the single JobTracker taking care of thousands of jobs and doing a lot of work alone.

In order to solve these issues, Facebook created Corona, which separated cluster resource management from job coordination.

In Hadoop Corona, the cluster resources are tracked by a central Cluster Manager. Each job gets its own Corona Job Tracker which tracks just that particular job.

Corona has entirely redesigned MapReduce architecture to bring better cluster utilization and job scheduling, just like YARN did.

Facebook's goals in re-writing the Hadoop scheduling framework were not the same as YARN's. Facebook wanted quick improvements in MapReduce, but only the part that they were...