Book Image

Mastering Mesos

By : Dipa Dubhashi, Akhil Das
Book Image

Mastering Mesos

By: Dipa Dubhashi, Akhil Das

Overview of this book

Apache Mesos is open source cluster management software that provides efficient resource isolations and resource sharing distributed applications or frameworks. This book will take you on a journey to enhance your knowledge from amateur to master level, showing you how to improve the efficiency, management, and development of Mesos clusters. The architecture is quite complex and this book will explore the difficulties and complexities of working with Mesos. We begin by introducing Mesos, explaining its architecture and functionality. Next, we provide a comprehensive overview of Mesos features and advanced topics such as high availability, fault tolerance, scaling, and efficiency. Furthermore, you will learn to set up multi-node Mesos clusters on private and public clouds. We will also introduce several Mesos-based scheduling and management frameworks or applications to enable the easy deployment, discovery, load balancing, and failure handling of long-running services. Next, you will find out how a Mesos cluster can be easily set up and monitored using the standard deployment and configuration management tools. This advanced guide will show you how to deploy important big data processing frameworks such as Hadoop, Spark, and Storm on Mesos and big data storage frameworks such as Cassandra, Elasticsearch, and Kafka.
Table of Contents (16 chapters)
Mastering Mesos
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Preface
Index

The attributes and resources of Mesos


Mesos describes the slave nodes present in the cluster by the following two methods:

Attributes

Attributes are used to describe certain additional information regarding the slave node, such as its OS version, whether it has a particular type of hardware, and so on. They are expressed as key-value pairs with support for three different value types—scalar, range, and text—that are sent along with the offers to frameworks. Take a look at the following code:

attributes : attribute ( ";" attribute )*

attribute : text ":" ( scalar | range | text )

Resources

Mesos can manage three different types of resources: scalars, ranges, and sets. These are used to represent the different resources that a Mesos slave has to offer. For example, a scalar resource type could be used to represent the amount of CPU on a slave. Each resource is identified by a key string, as follows:

resources : resource ( ";" resource )*

resource : key ":" ( scalar | range | set )

key : text ( "(" resourceRole ")" )?

resourceRole : text | "*"

Predefined uses and conventions

The Mesos master predefines how it handles the following list of resources:

  • cpus

  • mem

  • disk

  • ports

In particular, a slave without the cpu and mem resources will never have its resources advertised to any frameworks. Also, the master's user interface interprets the scalars in mem and disk in terms of MB. For example, the value 15000 is displayed as 14.65GB.

Examples

Here are some examples of configuring the Mesos slaves:

  • resources='cpus:24;mem:24576;disk:409600;ports:[21000-24000];bugs:{a,b,c}'

  • attributes='rack:abc;zone:west;os:centos5;level:10;keys:[1000-1500]'

In this case, we have three different types of resources, scalars, a range, and a set. They are called cpus, mem, and disk, and the range type is ports.

  • A scalar called cpus with the value 24

  • A scalar called mem with the value 24576

  • A scalar called disk with the value 409600

  • A range called ports with values 21000 through 24000 (inclusive)

  • A set called bugs with the values a, b, and c

In the case of attributes, we will end up with three attributes:

  • A rack attribute with the text value abc

  • A zone attribute with the text value west

  • An os attribute with the text value centos5

  • A level attribute with the scalar value 10

  • A keys attribute with range values 1000 through 1500 (inclusive)