Book Image

Mastering Mesos

By : Dipa Dubhashi, Akhil Das
Book Image

Mastering Mesos

By: Dipa Dubhashi, Akhil Das

Overview of this book

Apache Mesos is open source cluster management software that provides efficient resource isolations and resource sharing distributed applications or frameworks. This book will take you on a journey to enhance your knowledge from amateur to master level, showing you how to improve the efficiency, management, and development of Mesos clusters. The architecture is quite complex and this book will explore the difficulties and complexities of working with Mesos. We begin by introducing Mesos, explaining its architecture and functionality. Next, we provide a comprehensive overview of Mesos features and advanced topics such as high availability, fault tolerance, scaling, and efficiency. Furthermore, you will learn to set up multi-node Mesos clusters on private and public clouds. We will also introduce several Mesos-based scheduling and management frameworks or applications to enable the easy deployment, discovery, load balancing, and failure handling of long-running services. Next, you will find out how a Mesos cluster can be easily set up and monitored using the standard deployment and configuration management tools. This advanced guide will show you how to deploy important big data processing frameworks such as Hadoop, Spark, and Storm on Mesos and big data storage frameworks such as Cassandra, Elasticsearch, and Kafka.
Table of Contents (16 chapters)
Mastering Mesos
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Preface
Index

Resource isolation


One of the key requirements of a cluster manager is to ensure that the allocation of resources to a particular framework does not have an impact on any active running jobs of some other framework. Provision for isolation mechanisms on slaves to compartmentalize different tasks is thus a key feature of Mesos. Containers are leveraged for resource isolation with a pluggable architecture. The Mesos slave uses the Containerizer API to provide an isolated environment to run a framework's executor and its corresponding tasks. The Containerizer API's objective is to support a wide range of implementations, which implies that custom containerizers and isolators can be developed. When a slave process starts, the containerizer to be used to launch containers and a set of isolators to enforce the resource constraints can be specified.

The Mesos Containerizer API provides a resource isolation of framework executors using Linux-specific functionality, such as control groups and namespaces. It also provides basic support for POSIX systems (only resource usage reporting and not actual isolation). This important topic will be explored at length in subsequent chapters.

Mesos also provides network isolation at a container level to prevent a single framework from capturing all the available network bandwidth or ports. This is not supported by default, however, and additional dependencies need to be installed and configured in order to activate this feature.