Book Image

Apache Mesos Cookbook

By : David Blomquist, Tomasz Janiszewski
Book Image

Apache Mesos Cookbook

By: David Blomquist, Tomasz Janiszewski

Overview of this book

Apache Mesos is open source cluster sharing and management software. Deploying and managing scalable applications in large-scale clustered environments can be difficult, but Apache Mesos makes it easier with efficient resource isolation and sharing across application frameworks. The goal of this book is to guide you through the practical implementation of the Mesos core along with a number of Mesos supported frameworks. You will begin by installing Mesos and then learn how to configure clusters and maintain them. You will also see how to deploy a cluster in a production environment with high availability using Zookeeper. Next, you will get to grips with using Mesos, Marathon, and Docker to build and deploy a PaaS. You will see how to schedule jobs with Chronos. We’ll demonstrate how to integrate Mesos with big data frameworks such as Spark, Hadoop, and Storm. Practical solutions backed with clear examples will also show you how to deploy elastic big data jobs. You will find out how to deploy a scalable continuous integration and delivery system on Mesos with Jenkins. Finally, you will configure and deploy a highly scalable distributed search engine with ElasticSearch. Throughout the course of this book, you will get to know tips and tricks along with best practices to follow when working with Mesos.
Table of Contents (15 chapters)
Title Page
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Reconciliation


In this recipe, we will learn how to reconcile the state between the framework and Mesos.

How to do it...

Although our tasks persist after the framework restarts, they cannot be managed because the framework does not know about them. To make it aware of tasks it had scheduled previously, we need to keep them between restarts. We will do it in the same way as framework info. We will store the task state after every update and load it after registration.

Declare the global variable with the path to task state file:

var stateFile = fmt.Sprintf("%s/%s", os.TempDir(), "state.json")

In the handle update function, just after updating the task state, save it to the file:

stateJSON, _ := json.Marshal(tasksState)
ioutil.WriteFile(stateFile, stateJSON, 0644)

We will load the previous state in the dedicated function and that should be called after the frameworkId is stored in the framework info:

func reconcile() {
      oldState, err := ioutil.ReadFile(stateFile)
      if err == nil {
      ...