Book Image

Apache Mesos Cookbook

By : David Blomquist, Tomasz Janiszewski
Book Image

Apache Mesos Cookbook

By: David Blomquist, Tomasz Janiszewski

Overview of this book

Apache Mesos is open source cluster sharing and management software. Deploying and managing scalable applications in large-scale clustered environments can be difficult, but Apache Mesos makes it easier with efficient resource isolation and sharing across application frameworks. The goal of this book is to guide you through the practical implementation of the Mesos core along with a number of Mesos supported frameworks. You will begin by installing Mesos and then learn how to configure clusters and maintain them. You will also see how to deploy a cluster in a production environment with high availability using Zookeeper. Next, you will get to grips with using Mesos, Marathon, and Docker to build and deploy a PaaS. You will see how to schedule jobs with Chronos. We’ll demonstrate how to integrate Mesos with big data frameworks such as Spark, Hadoop, and Storm. Practical solutions backed with clear examples will also show you how to deploy elastic big data jobs. You will find out how to deploy a scalable continuous integration and delivery system on Mesos with Jenkins. Finally, you will configure and deploy a highly scalable distributed search engine with ElasticSearch. Throughout the course of this book, you will get to know tips and tricks along with best practices to follow when working with Mesos.
Table of Contents (15 chapters)
Title Page
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Introduction


Apache Mesos is cluster management software that distributes the combined resources of many individual servers to applications through frameworks. Mesos is open source software that is free to download and use in accordance with the Apache License 2.0. This chapter will provide the reader with recipes for installing and configuring Apache Mesos.

Mesos can run on Linux, Mac, and Windows. However, we recommend running Mesos on Linux for production deployments and on Mac and Windows for development purposes only. Mesos can be installed from source code or from binary packages downloaded from repositories. We will cover a few installation methods on select operating systems in this chapter. The reasons for covering these specific operating systems and installation methods are as follows:

  • The operating systems natively include a kernel that supports full resource isolation
  • The operating systems are current as of this writing with long-term support
  • The operating systems and installation methods do not require workarounds or an excessive number of external repositories
  • The installation methods use the latest stable version of Mesos, whether it is from binary packages or source code

Based on these criteria, we have covered the following installation methods for this book:

  • Mesosphere packages on Ubuntu 16.04 LTS
  • Mesosphere packages on Ubuntu 14.04 LTS
  • Mesosphere packages on CentOS 7 and RHEL 7
  • Source code on Ubuntu 16.04 LTS
  • Source code on Ubuntu 14.04 LTS
  • Source code on OS X (Yosemite and El Capitan)

Mesosphere, a company founded by one of the original developers of Mesos, provides free, open source binary packages as well as commercial support for Mesos. Mesosphere binary packages are well maintained and provide an easy way to install and run Mesos. We use Mesosphere packages installed on Ubuntu 14.04 as the base environment for the recipes in chapters 3 - 8 and we recommend you use that installation method with this book. If you need to run Mesos one of the other supported operating systems, we provide you with the installation instructions in this chapter but you will need to adapt the recipes in chapters 3 - 8 for use with your operating system. Installing Mesos from source will allow you to customize the build and install process and enable and disable features so we will also guide you through a source install in this chapter. If you want a development environment on a Mac, building from source on OS X is the only way to go.

We will guide you through the installations in the following sections but first, you will need to plan your Mesos deployment. For a Mesos development environment, you only need one host or node. The node can be a physical computer, a virtual machine, or a cloud instance. For a production cluster, we recommend at least three master nodes and as many slave nodes as you will need to support your application frameworks. You can think of the slave nodes as a pool of CPU, RAM, and storage that can be increased by simply adding more slave nodes. Mesos makes it very easy to add slave nodes to an existing cluster as your application requirements grow. At this point, you should determine whether you will be building a Mesos development environment or a production cluster and you should have an idea of how many master and slave nodes you will need. The next sections will provide recipes for installing Mesos in the environment of your choice.