Book Image

Apache Mesos Essentials

By : Dharmesh Kakadia
Book Image

Apache Mesos Essentials

By: Dharmesh Kakadia

Overview of this book

<p>Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It allows developers to concurrently run the likes of Hadoop, Spark, Storm, and other applications on a dynamically shared pool of nodes. With Mesos, you have the power to manage a wide range of resources in a multi-tenant environment.</p> <p>Starting with the basics, this book will give you an insight into all the features that Mesos has to offer. You will first learn how to set up Mesos in various environments from data centers to the cloud. You will then learn how to implement self-managed Platform as a Service environment with Mesos using various service schedulers, such as Chronos, Aurora, and Marathon. You will then delve into the depths of Mesos fundamentals and learn how to build distributed applications using Mesos primitives.</p> <p>Finally, you will round things off by covering the operational aspects of Mesos including logging, monitoring, high availability, and recovery.</p>
Table of Contents (15 chapters)
Apache Mesos Essentials
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Hadoop on Mesos


Running Hadoop on Mesos allows you to share cluster resources with other frameworks. By running Hadoop on Mesos, we can leverage the huge community and frameworks built on top of Hadoop, such as Giraph, Hama, HBase, Hive, and so on. Also, if we are already using Hadoop, by running it on Mesos, we can continue using tools built around the Hadoop ecosystem, but with improved resource utilization. Existing MapReduce code and tools will continue working with Hadoop on Mesos. With Hadoop on Mesos, we can also run multiple versions of Hadoop on the same Mesos cluster resources. Hadoop on the Mesos project implements a Mesos framework scheduler around Hadoop's JobTracker and wraps Hadoop executors to Mesos executors. The following figure shows you how Hadoop and Mesos integrate.

In the following figure, the master node is running JobTracker on the Mesos master node and some Mesos slaves are running TaskTrackers. Each TaskTracker is running some Map and Reduce task slots. Note that...