Book Image

Mastering Mesos

By : Dipa Dubhashi, Akhil Das
Book Image

Mastering Mesos

By: Dipa Dubhashi, Akhil Das

Overview of this book

Apache Mesos is open source cluster management software that provides efficient resource isolations and resource sharing distributed applications or frameworks. This book will take you on a journey to enhance your knowledge from amateur to master level, showing you how to improve the efficiency, management, and development of Mesos clusters. The architecture is quite complex and this book will explore the difficulties and complexities of working with Mesos. We begin by introducing Mesos, explaining its architecture and functionality. Next, we provide a comprehensive overview of Mesos features and advanced topics such as high availability, fault tolerance, scaling, and efficiency. Furthermore, you will learn to set up multi-node Mesos clusters on private and public clouds. We will also introduce several Mesos-based scheduling and management frameworks or applications to enable the easy deployment, discovery, load balancing, and failure handling of long-running services. Next, you will find out how a Mesos cluster can be easily set up and monitored using the standard deployment and configuration management tools. This advanced guide will show you how to deploy important big data processing frameworks such as Hadoop, Spark, and Storm on Mesos and big data storage frameworks such as Cassandra, Elasticsearch, and Kafka.
Table of Contents (16 chapters)
Mastering Mesos
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Preface
Index

Deploying and configuring a Mesos cluster using Cloudformation


In this module, we will discuss how we can use Cloudformation scripts to launch a Mesos cluster on Amazon AWS. Before jumping into it, first make sure you install and configure aws-cli on the machines where you want to launch the cluster. Take a look at the instructions from the following repository to set up aws-cli:

https://github.com/aws/aws-cli.

The next thing that we need after setting up aws-cli is the cloudformation-zookeeper template for an exhibitor-managed ZooKeeper cluster.

Setting up cloudformation-zookeeper

We first need to clone the following repository as it contains the JSON file that has the parameters, descriptors, and configuration values:

$ git clone https://github.com/mbabineau/cloudformation-zookeeper

Log in to the AWS console and open up the following ports for security group:

  • SSH Port: 22

  • ZooKeeper Client Port: 2181

  • Exhibitor HTTP Port: 8181

We can now use aws-cli to launch the cluster using the following command...