Book Image

Native Docker Clustering with Swarm

By : Fabrizio Soppelsa, Chanwit Kaewkasi
Book Image

Native Docker Clustering with Swarm

By: Fabrizio Soppelsa, Chanwit Kaewkasi

Overview of this book

Docker Swarm serves as one of the crucial components of the Docker ecosystem and offers a native solution for you to orchestrate containers. It’s turning out to be one of the preferred choices for Docker clustering thanks to its recent improvements. This book covers Swarm, Swarm Mode, and SwarmKit. It gives you a guided tour on how Swarm works and how to work with Swarm. It describes how to set up local test installations and then moves to huge distributed infrastructures. You will be shown how Swarm works internally, what’s new in Swarmkit, how to automate big Swarm deployments, and how to configure and operate a Swarm cluster on the public and private cloud. This book will teach you how to meet the challenge of deploying massive production-ready applications and a huge number of containers on Swarm. You'll also cover advanced topics that include volumes, scheduling, a Libnetwork deep dive, security, and platform scalability.
Table of Contents (18 chapters)
Native Docker Clustering with Swarm
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Dedication
Preface

Swarm, yesterday


For the very first example, we'll run the easiest possible configuration of a Swarm v1 cluster locally to get a taste of how "old" Swarms worked (and still works). This tiny cluster will have the following features:

  • Made of four nodes of 1CPU, 1GB of memory each, it will consist of an infrastructure of four CPUs and 4GB of memory available in total

  • Each node will run on VirtualBox

  • Each node is connected to each other on the local VirtualBox network

  • No discovery service is involved: a static nodes:// mechanism will be used

  • No security is configured, in other words TLS is disabled

Our cluster will look similar to the following diagram. Four Engines will be connected to each other through port 3376 in a mesh. Beyond the Docker engine, in fact, each of them will run a Docker container exposing port 3376 (Swarm) on host and redirecting it into itself. We, operators, will be able to connect to (any of) the hosts by setting the environment variable DOCKER_HOST to IP:3376. Everything will become clearer if you follow the example step-by-step.

To begin, we must create four Docker hosts with Docker Machine. Docker Machine automates these steps with one command instead of manually creating a Linux VM, generating and uploading certificates, logging into it via SSH, and installing and configuring the Docker daemon.

Machine will perform the following steps:

  1. Spin up a VirtualBox VM starting from the boot2docker image.

  2. Assign the VM an IP on the VirtualBox internal network.

  3. Upload and configure certificates and keys.

  4. Install Docker daemon on this VM.

  5. Configure the Docker daemon and expose it so it will be remotely reachable.

As a result, we'll have a VM running Docker and ready to be accessed to run containers.

Boot2Docker

Built with Tiny Core Linux, Boot2Docker is a lightweight distribution, which is especially designed for running Docker containers. It's runs completely on RAM and the boot time is extremely fast, around five seconds from start to the console. When starting the Engine, Boot2Docker starts Docker Engine at the secure port 2376 by default.

Boot2Docker is by no mean for the production workload. It's designed for development and testing purpose only. We'll start with using boot2docker then later move on to the production in subsequent chapters. At the time of writing, Boot2Docker supports Docker 1.12.3 and uses Linux Kernel 4.4. It comes with AUFS 4 as the default storage driver for Docker Engine.

Create 4 cluster nodes with Docker Machine

If we execute:

$ docker-machine ls

on our new installation to list the available machines, we see that we have no running ones.

So, let's start by creating one, with this command:

$ docker-machine create --driver virtualbox node0

This command specifically asks to use the VirtualBox driver (-d for short) and to name the machine node0. Docker Machines can provision machines on dozens of different public and private providers, such as AWS, DigitalOcean, Azure, OpenStack, and has lots of options. For now, we go with the standard settings. The first cluster node will be ready after some time.

At this point, issue the following command to get a control of this host (so to remotely gain access):

$ docker-machine env node0

This will print some shell variables. Just copy the last row, the one with eval, paste it and issue enter. With those variables configured, you are not operating the local daemon anymore (if any), but the Docker daemon of node0.

If you check the list of machines again, you will see a * next to the image name, to indicate that it's the machine currently in use. Alternatively, you can type the following command to print the currently active machine:

$ docker-machine active

The daemon is running on this machine, with some standard settings (such as on port tcp/2376 enabled TLS). You can ensure that by SSHing to the node and verify the running processes:

$ docker-machine ssh node0 ps aux | grep docker
1320 root  /usr/local/bin/docker daemon -D -g /var/lib/docker -H 
    unix:// -H tcp://0.0.0.0:2376 --label provider=virtualbox --
    tlsverify --tlscacert=/var/lib/boot2docker/ca.pem -- 
    tlscert=/var/lib/boot2docker/server.pem -- 
    tlskey=/var/lib/boot2docker/server-key.pem -s aufs

So, you can immediately this Docker daemon by, for example, starting containers and checking the Docker status:

Perfect! Now we provision the other three hosts, in the same exact way, by naming them node1, node2, and node3:

$ docker-machine create --driver virtualbox node1
$ docker-machine create --driver virtualbox node2
$ docker-machine create --driver virtualbox node3

When they finish, you will have four Docker hosts available. Check with Docker machine.

We're now ready to start a Swarm cluster. But, before, for this very first example in order to keep it as simple as possible, we'll go with disabling TLS for running engines. Our plan is: Run the Docker daemon on port 2375, without TLS.

Let's make a bit of order and explain all ports combinations in detail.

Insecure

Secure

Engine: 2375

Engine: 2376

Swarm: 3375

Swarm: 3376

Swarm v2 uses 2377 for node discovery among nodes

Port 2377 is for Swarm v2 node to discover each other nodes in the cluster.

Configuring the Docker hosts

To understand where the TLS configuration is, we'll do some exercises by turning off the TLS of all our Docker hosts. Also turning it off here is intended to motivate the readers to learn how the swarm manage command works by invoking it ourselves.

We have four hosts running Docker on port tcp/2376 and with TLS, as Docker Machine creates them by default. We must reconfigure them to change the daemon port to tls/2375 and remove TLS. So, we log in into each of them, with this command:

$ docker-machine ssh node0

Then, we gain root privileges:

$ sudo su -

And configure boot2docker, by modifying the file /var/lib/boot2docker/profile:

# cp /var/lib/boot2docker/profile /var/lib/boot2docker/profile-bak
# vi /var/lib/boot2docker/profile

We delete the rows with CACERT, SERVERKEY, and SERVERCERT and configure the daemon port to tcp/2375 and DOCKER_TLS to no. In practice this will be our configuration:

After this log out from the SSH session and restart the machine:

$ docker-machine restart node0

Docker is now running on port tcp/2375 with no security. You can check this with the following command:

$ docker-machine ssh node0 ps aux | grep docker
 1127 root  /usr/local/bin/docker daemon -D -g /var/lib/docker -H 
     unix:// -H tcp://0.0.0.0:2375 --label provider=virtualbox -s aufs

Finally, on your local desktop computer, unset DOCKER_TLS_VERIFY and re-export DOCKER_HOST in order to use the daemon listening on tcp/2375 with no TLS:

$ unset DOCKER_TLS_VERIFY
$ export DOCKER_HOST="tcp://192.168.99.103:2375" 

We must repeat these steps for each of our four nodes that will be part of our first Swarm.

Starting Docker Swarm

To get started with Swarm v1 (no surprise), one must pull the swarm image from the Docker hub. Open the four terminals, source the environment variables for each of your machines in each one in the first one, source node0 (docker-machine env node0, and copy and paste the env variable to the shell), in second node1, and so on -, and after completing the steps for changing the standard port and disabling TLS described some lines above, on each of them do:

$ docker pull swarm

We'll use no discovery service for the first example, but the simplest of the mechanisms, such as the nodes://. With nodes://, the Swarm cluster nodes are connected manually, to form a grid of peers. What the operator has to do is simply define a list of nodes IPs and the daemon port, separated by commas, as shown:

nodes://192.168.99.101:2375,192.168.99.102:2375,192.168.99.103:2375,192.168.99.107:2375

To use Swarm, you simply run the swarm container with some arguments. To show the help online, you type:

$ docker run swarm --help

As you see, Swarm has basically four commands:

  • Create is used to create clusters with a discovery service, for example token://

  • List shows the list of the cluster nodes

  • Manage allows you to operate a Swarm cluster

  • Join, in combination with a discovery service, is used for joining new nodes to an existing cluster

For now, we'll use the manage command. This is the command with most of the options (which you can investigate by issuing docker run swarm manage --help). We limit now to connect nodes. The following is the strategy on each node:

  1. Expose the Swarm service through the swarm container.

  2. Run this container in daemon (-d) mode.

  3. Forward the standard Swarm port tcp/3376 to the internal (on container) port tcp/2375.

  4. Specify the list of hosts part of the cluster, with nodes:// - each host has to be a pair IP:port where the port is the Docker engine port (tcp/2375).

So, in each terminal you're connected to every machine, execute this:

$ docker run \
-d \
-p 3376:2375 \
swarm manage \ 
    nodes://192.168.99.101:2375,192.168.99.102:2375,
    192.168.99.103:2375,192.168.99.107:2375

Tip

When using the nodes:// mechanism, you can use Ansible-like host range patterns, so compact the syntax of three contiguous IPs like nodes://192.168.99.101:2375,192.168.99.102:2375,192.168.99.103:2375 In nodes://192.168.99.[101:103]:2375

Now, as the next step, we'll connect to it and inspect its information before starting using for running containers. For convenience, open a new terminal. We connect now not anymore to the Docker engine on one of our nodes, but to the Docker Swarm. So we will connect to tcp/3376 and not to tcp/2375 anymore. For the purpose of showing in detail what we're doing, let's start by sourcing node0 variables:

$ docker-machine env node0

Copy and paste the eval line, as you already know, and check what shell variables are exported with the following command:

$ export | grep DOCKER_

We now need to do the following:

  1. Change the DOCKER_HOST to connect to Swarm port tcp/3376 instead of Engine tcp/2375

  2. Disable DOCKER_TLS_VERIFY.

  3. Disable DOCKER_CERT_PATH.

You should have a configuration similar to this:

If we now connect to the Docker swarm at 3376, and show some info, we see that we're running Swarm:

Congratulations! You just started your first Docker cluster with Swarm. We can see that we still have no containers running on our cluster apart from the four swarms, but the Server Version is swarm/1.2.3, the scheduling strategy is spread, and, most importantly, we have four healthy nodes in our swarm (details of each Swarm node follow).

Also, you can get some extra information regarding the scheduler behavior of this Swarm cluster:

Strategy: spread
Filters: health, port, containerslots, dependency, affinity, 
    constraint

A spread scheduling strategy means that Swarm will attempt to place containers on the less utilized host and the listed filters are available when you create containers, thus allowing you to decide to manually suggest some options. For example, you might want to make your Galera cluster containers geographically near but on different hosts.

But, what's the size of this Swarm? You can see it at the very end of this output:

It means that on this tiny Swarm you have the total availability of these resources: four CPUs and 4GB of memory. That's just what we expected, by merging the computational resources of 4 VirtualBox hosts with a CPU and 1GB of memory each.