Book Image

Native Docker Clustering with Swarm

By : Fabrizio Soppelsa, Chanwit Kaewkasi
Book Image

Native Docker Clustering with Swarm

By: Fabrizio Soppelsa, Chanwit Kaewkasi

Overview of this book

Docker Swarm serves as one of the crucial components of the Docker ecosystem and offers a native solution for you to orchestrate containers. It’s turning out to be one of the preferred choices for Docker clustering thanks to its recent improvements. This book covers Swarm, Swarm Mode, and SwarmKit. It gives you a guided tour on how Swarm works and how to work with Swarm. It describes how to set up local test installations and then moves to huge distributed infrastructures. You will be shown how Swarm works internally, what’s new in Swarmkit, how to automate big Swarm deployments, and how to configure and operate a Swarm cluster on the public and private cloud. This book will teach you how to meet the challenge of deploying massive production-ready applications and a huge number of containers on Swarm. You'll also cover advanced topics that include volumes, scheduling, a Libnetwork deep dive, security, and platform scalability.
Table of Contents (18 chapters)
Native Docker Clustering with Swarm
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Dedication
Preface

The Spark example, again


We're going to rearchitect the example of Chapter 6, Deploy Real Applications on Swarm, so we'll deploy Spark on Swarm, but this time with a realistic networking and storage setup.

Spark storage backend usually runs on Hadoop, or on NFS when on filesystem. For jobs not requiring storage, Spark will create local data on workers, but for storage computations, you will need a shared filesystem on each node, which cannot be guaranteed automatically by Docker volume plugins (at least, so far).

A possibility to achieve that goal on Swarm is to create NFS shares on each Docker host, and then mount them transparently inside service containers.

Our focus here is not to illustrate Spark job details and their storage organization, but to introduce an opinionated storage option for Docker and give an idea of how to organize and scale a fairly-complex service on Docker Swarm.