Understanding the Scheduler API | Apache Mesos Cookbook

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents

Apache Mesos Cookbook

By : Marco Massenzio, Blomquist, Tomasz Janiszewski

1 (1)

Apache Mesos Cookbook

1 (1)

By: Marco Massenzio, Blomquist, Tomasz Janiszewski

Overview of this book

Apache Mesos is open source cluster sharing and management software. Deploying and managing scalable applications in large-scale clustered environments can be difficult, but Apache Mesos makes it easier with efficient resource isolation and sharing across application frameworks. The goal of this book is to guide you through the practical implementation of the Mesos core along with a number of Mesos supported frameworks. You will begin by installing Mesos and then learn how to configure clusters and maintain them. You will also see how to deploy a cluster in a production environment with high availability using Zookeeper. Next, you will get to grips with using Mesos, Marathon, and Docker to build and deploy a PaaS. You will see how to schedule jobs with Chronos. We’ll demonstrate how to integrate Mesos with big data frameworks such as Spark, Hadoop, and Storm. Practical solutions backed with clear examples will also show you how to deploy elastic big data jobs. You will find out how to deploy a scalable continuous integration and delivery system on Mesos with Jenkins. Finally, you will configure and deploy a highly scalable distributed search engine with ElasticSearch. Throughout the course of this book, you will get to know tips and tricks along with best practices to follow when working with Mesos.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Sections

Conventions

Reader feedback

Customer support

Free Chapter

Getting Started with Apache Mesos

Getting Started with Apache Mesos

Introduction

Installing Mesos on Ubuntu 16.04 from packages

Installing Mesos on Ubuntu 14.04 from packages

Installing Mesos on CentOS 7 and RHEL 7 from packages

Preparing Ubuntu 16.04 for a Mesos installation from source code

Preparing Ubuntu 14.04 for a Mesos installation from source code

Preparing OS X (Yosemite and El Capitan) for a Mesos Installation from source code

Downloading, building, and installing the Mesos source code

Implementing High Availability with Apache ZooKeeper

Implementing High Availability with Apache ZooKeeper

Introduction

Configuring ZooKeeper installed from packages on Ubuntu 14.04, Ubuntu 16.04, CentOS 7, or RHEL 7

Configuring ZooKeeper on Ubuntu 14.04, 16.04, and OS X from source

Running and Maintaining Mesos

Running and Maintaining Mesos

Introduction

Logging and debugging

Monitoring

Setting attributes for agents

Defining roles and resources

Oversubscription

Controlling permissions using access control lists

Agent authentication

Encryption SSL

Upgrading Mesos

Understanding the Scheduler API

Understanding the Scheduler API

Introduction

Installing Protobuf

Registering frameworks

Handling events

Declining offers

Scheduling tasks

Acknowledging task updates

Killing tasks

State persistence

Reconciliation

Managing Containers

Managing Containers

Introduction

Enabling the Mesos containerizer

Enabling POSIX isolators

Enabling the POSIX disk isolator

Configuring the shared filesystem isolator

Configuring cgroup isolators

Configuring the port mapping network isolator

Configuring Docker image support for the Mesos containerizer

Using the Docker containerizer

Running an image from a private repository

Using container network interface

Monitoring containers with Sysdig

Deploying PaaS with Marathon

Deploying PaaS with Marathon

Introduction

Installing and configuring Marathon

Monitoring Marathon

Launching an application

Launching a Docker container

Setting up the discovery service

Enabling SSL

Enabling basic access authentication

Using a custom authentication plugin

Tuning Marathon

Job Scheduling with Metronome

Job Scheduling with Metronome

Introduction

Installing and configuring Metronome

Monitoring Metronome

Scheduling jobs

Continuous Integration with Jenkins

Continuous Integration with Jenkins

Introduction

Building the Jenkins Mesos plugin

Installing Jenkins

Enabling the Jenkins Mesos plugin

Declining offers

In this recipe, we will handle offers by declining them so they can be presented to other frameworks.

How to do it...

So far, our framework has been acquiring resources, although it's not using them. This is because it gets offers but does not respond to them. To make offers free, we need to explicitly refuse them.

To refuse Offers, we need to handle the Offers event. We can do this by adding the following code in the event type switch:

case Event_OFFERS:
         log.Printf("Handle offers returns: %v", handleOffers(event.Offers))

Of course, we need to implement the handleOffers() function. But before that, we should prepare the call() function, which will be useful for implementation. This function will be used to send messages to Mesos:

func call(message *Call) error {
        message.FrameworkId = frameworkInfo.Id
        body, _ := marshaller.MarshalToString(message)
        req, _ := http.NewRequest("POST", schedulerApiUrl, bytes.NewBuffer([]byte(body)))
        req.Header...

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Apache Mesos Cookbook

Search

Your notes and bookmarks