Book Image

YARN Essentials

Book Image

YARN Essentials

Overview of this book

If you have a working knowledge of Hadoop 1.x but want to start afresh with YARN, this book is ideal for you. You will be able to install and administer a YARN cluster and also discover the configuration settings to fine-tune your cluster both in terms of performance and scalability. This book will help you develop, deploy, and run multiple applications/frameworks on the same shared YARN cluster.
Table of Contents (12 chapters)
Free Chapter
1
1. Need for YARN
9
9. YARN – Alternative Solutions
11
Index

KOYA (Kafka on YARN)


On November 5, 2014, DataTorrent, a company founded by ex-Yahoo!, announced a new project to bring the fault-tolerant, high-performance, scalable Apache Kafka messaging system to YARN.

The so-called Kafka on YARN (KOYA) project plans to leverage YARN for Kafka broker management, automatic broker recovery, and more. Planned features include a fully-HA ApplicationMaster, sticky allocation of containers (so that a restart can access local data), a web interface for Kafka, and more.

The expected release to the open source community is somewhere in Q2 2015.

More information is available at https://www.datatorrent.com/introducing-koya-apache-kafka-on-apache-hadoop-2-0-yarn/.