Book Image

Native Docker Clustering with Swarm

By : Fabrizio Soppelsa, Chanwit Kaewkasi
Book Image

Native Docker Clustering with Swarm

By: Fabrizio Soppelsa, Chanwit Kaewkasi

Overview of this book

Docker Swarm serves as one of the crucial components of the Docker ecosystem and offers a native solution for you to orchestrate containers. It’s turning out to be one of the preferred choices for Docker clustering thanks to its recent improvements. This book covers Swarm, Swarm Mode, and SwarmKit. It gives you a guided tour on how Swarm works and how to work with Swarm. It describes how to set up local test installations and then moves to huge distributed infrastructures. You will be shown how Swarm works internally, what’s new in Swarmkit, how to automate big Swarm deployments, and how to configure and operate a Swarm cluster on the public and private cloud. This book will teach you how to meet the challenge of deploying massive production-ready applications and a huge number of containers on Swarm. You'll also cover advanced topics that include volumes, scheduling, a Libnetwork deep dive, security, and platform scalability.
Table of Contents (18 chapters)
Native Docker Clustering with Swarm
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Dedication
Preface

Swarm, today


In this section, we'll set up a small cluster with the new Swarm mode built in Docker Engine 1.12 or later.

At the DockerCon16, among the big announcements, two drew big attention regarding containers orchestration:

  • The integration between the Engine and Swarm, called the Docker Swarm mode.

  • SwarmKit

In practice, the Docker daemon, starting from version 1.12, adds the possibility to run a so-called Swarm Mode. New CLI commands were added to the docker client, such as node, service, stack, deploy, alongside with, of course, swarm.

We'll cover Swarm Mode and SwarmKit in more detail starting fromChapter 3, Meeting Docker Swarm Mode, but now that we finished the example with Swarm v1, we're going to give the reader a taste on how Swarm v2 has a much simpler user experience than v1. The only requirement to use Swarm v2 is to have a daemon version of at least version 1.12-rc1. But with Docker Machine 0.8.0-rc1+, you can provision Docker hosts fulfilling this requirement with the usual procedure.

Docker also announced Docker for AWS and Docker for Azure at DockerCon 2016. Not only AWS and Azure, but actually we're also fans of DigitalOcean, so we created a new tool that wraps around doctl the DigitalOcean command line interface, to help provision Docker cluster in the new massively way. The tool is called belt and now available from http://github.com/chanwit/belt. You can pull belt with this command:

go get github.com/chanwit/belt

or download the binary from the Release tab of the project.

First, we'll prepare a template file for provisioning on DigitalOcean. Your .belt.yaml will look like this:

$ cat .belt.yaml
---
digitalocean:
  region: sgp1
  image: 18153887
  ssh_user: root
  ssh_key_fingerprint: 816630

Please note that my image number 18153887 is the snapshot containing Docker 1.12. DigitalOcean usually makes the latest Docker image available after every release. To make you able to control your cluster, SSH key needs to be there. For the field ssh_key_fingerprint, you can either put the finger print as well as the key ID.

Do not forget to set your DIGITALOCEAN_ACCESS_TOKEN environment variable. Also, Belt recognizes the same set of Docker Machine shell variables. If you are familiar with Docker Machine you'll know how to set them. To refresh, these are the shell variables we introduced in the previous section:

  • export DOCKER_TLS_VERIFY="1"

  • export DOCKER_HOST="tcp://<IP ADDRESS>:2376"

  • export DOCKER_CERT_PATH="/Users/user/.docker/machine/machines/machine"

  • export DOCKER_MACHINE_NAME="machine"

So, now let's see how to use Belt:

$ export DIGITALOCEAN_ACCESS_TOKEN=1b207 .. snip .. b6581c

Now we create a Swarm of four nodes each with 512M of memory:

$ belt create 512mb node[1:4]
ID              Name    Public IPv4     Memory  VCPUs   Disk
18511682        node1                   512     1       20  
18511683        node4                   512     1       20  
18511684        node3                   512     1       20  
18511681        node2                   512     1       20  

You can see that we can specify a set of nodes with similar syntax node[1:4]. This command created four nodes on DigitalOcean. Please wait for about 55 seconds for all nodes to be provisioned. Then you can list them:

$ belt ls
ID              Name    Public IPv4       Status  Tags
18511681        node2   128.199.105.119   active
18511682        node1   188.166.183.86    active
18511683        node4   188.166.183.103   active
18511684        node3   188.166.183.157   active

Their status now has changed from "new" to "active". All IP addresses are assigned. Everything is good to go for now.

We now can start Swarm.

Before that make sure we are running Docker 1.12. We check this on node1.

$ belt active node1
node1
$ belt docker version
Client:
 Version:      1.12.0-rc2
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   906eacd
 Built:        Fri Jun 17 21:02:41 2016
 OS/Arch:      linux/amd64
 Experimental: true
Server:
 Version:      1.12.0-rc2
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   906eacd
 Built:        Fri Jun 17 21:02:41 2016
 OS/Arch:      linux/amd64
 Experimental: true

The belt docker command is just a thin wrapper command that sends the whole command line going through SSH to your Docker host. So this tool will not get in the way and your Docker Engines is always in-control.

Now we will initialize the first node with Swarm Mode.

$ belt docker swarm init
Swarm initialized: current node (c0llmsc5t1tsbtcblrx6ji1ty) is now 
    a manager.

Then we'll join other three nodes to this newly formed cluster. Joining a large cluster is a tedious task. Instead of going through every node and do docker swarm join manually, we'll let belt do this for us:

$ belt swarm join node1 node[2:4]
node3: This node joined a Swarm as a worker.
node2: This node joined a Swarm as a worker.
node4: This node joined a Swarm as a worker.

Tip

You can of course be able to run: belt --host node2 docker swarm join <node1's IP>:2377 to manually join node2 to your cluster.

And you'll get this view of cluster:

$ belt docker node ls
ID          NAME   MEMBERSHIP  STATUS  AVAILABILITY  MANAGER STATUS
4m5479vud9qc6qs7wuy3krr4u    node2  Accepted    Ready   Active
4mkw7ccwep8pez1jfeok6su2o    node4  Accepted    Ready   Active
a395rnht2p754w1beh74bf7fl    node3  Accepted    Ready   Active
c0llmsc5t1tsbtcblrx6ji1ty *  node1  Accepted    Ready   Active        Leader

Congratulations! You just installed a Swarm cluster on DigitalOcean.

We now create a service for nginx. This command creates an Nginx service with 2 instances of containers published at port 80.

$ belt docker service create --name nginx --replicas 2 -p 80:80 
    nginx
d5qmntf1tvvztw9r9bhx1hokd

Here we go:

$ belt docker service ls
ID            NAME   REPLICAS  IMAGE  COMMAND
d5qmntf1tvvz  nginx  2/2       nginx

Now let's scale it to 4 nodes.

$ belt docker service scale nginx=4
nginx scaled to 4
$ belt docker service ls
ID            NAME   REPLICAS  IMAGE  COMMAND
d5qmntf1tvvz  nginx  4/4       nginx

Similar to Docker Swarm, you can now use belt ip to see where the node runs. You can use any IP address to browse the NGINX service. It's available on every node.

$ belt ip node2
128.199.105.119

This is how Swarm Mode looks like starting from Docker 1.12.