Book Image

The DevOps 2.1 Toolkit: Docker Swarm

By : Viktor Farcic
Book Image

The DevOps 2.1 Toolkit: Docker Swarm

By: Viktor Farcic

Overview of this book

Viktor Farcic's latest book, The DevOps 2.1 Toolkit: Docker Swarm, takes you deeper into one of the major subjects of his international best seller, The DevOps 2.0 Toolkit, and shows you how to successfully integrate Docker Swarm into your DevOps toolset. Viktor shares with you his expert knowledge in all aspects of building, testing, deploying, and monitoring services inside Docker Swarm clusters. You'll go through all the tools required for running a cluster. You'll travel through the whole process with clusters running locally on a laptop. Once you're confident with that outcome, Viktor shows you how to translate your experience to different hosting providers like AWS, Azure, and DigitalOcean. Viktor has updated his DevOps 2.0 framework in this book to use the latest and greatest features and techniques introduced in Docker. We'll go through many practices and even more tools. While there will be a lot of theory, this is a hands-on book. You won't be able to complete it by reading it on the metro on your way to work. You'll have to read this book while in front of the computer and get your hands dirty.

Related Content you might be interested in

Current Title:

Small book image
The DevOps 2.1 Toolkit: Docker Swarm
Viktor Farcic
May, 2017 | 436 pages
Small book image
The DevOps 2.2 Toolkit
Viktor Farcic
Mar, 2018 | 360 pages
Small book image
The DevOps 2.4 Toolkit
Viktor Farcic
Nov, 2019 | 398 pages
Small book image
The DevOps 2.3 Toolkit
Viktor Farcic
Sep, 2018 | 418 pages
Table of Contents (22 chapters)
Title Page
Title Page
Credits
Credits
About the Author
About the Author
www.PacktPub.com
www.PacktPub.com
Customer Feedback
Customer Feedback
Preface
Preface
Free Chapter
1
Continuous Integration with Docker Containers
Continuous Integration with Docker Containers
Defining a fully Dockerized manual Continuous Integration flow
Running unit tests and building service binary
Building service images
Running staging tests
Pushing images to the registry
2
Setting Up and Operating a Swarm Cluster
Setting Up and Operating a Swarm Cluster
Scalability
Axis scaling
Clustering
Docker Swarm Mode
Setting up a Swarm cluster
Deploying services to the Swarm cluster
Scaling services
Failover
What now?
3
Docker Swarm Networking and Reverse Proxy
Docker Swarm Networking and Reverse Proxy
Setting up a cluster
Requirements of secured and fault tolerant services running with high availability
Running a database in isolation
Running a service through a reverse proxy
Creating a reverse proxy service in charge of routing requests depending on their base URLs
Load balancing requests across all instances of a service
What now?
4
Service Discovery inside a Swarm Cluster
Service Discovery inside a Swarm Cluster
What would Docker Swarm look like without?
What does standalone Docker Swarm look like with service discovery?
Service discovery in the Swarm cluster
Do we need service discovery?
Setting up Consul as service registry inside a Swarm cluster
Problems when scaling stateful instances
Using service registry to store the state
Discovering addresses of all instances that form a service
Using service registry or key value store to store service state
What now?
5
Continuous Delivery and Deployment with Docker Containers
Continuous Delivery and Deployment with Docker Containers
Defining the Continuous Delivery environment
Setting up Continuous Delivery clusters
Using node labels to constrain services
Creating services
Walking through Continuous Delivery steps
Walking the extra mile from Continuous Delivery to Continuous Deployment
What now?
6
Automating Continuous Deployment Flow with Jenkins
Automating Continuous Deployment Flow with Jenkins
Jenkins architecture
Production environment setup
Jenkins service
Jenkins failover
Jenkins agent
Creating services in production and production-like environments
Automating Continuous Deployment flow with Jenkins
Creating Jenkins Pipeline jobs
What now?
7
Exploring Docker Remote API
Exploring Docker Remote API
Setting up the environment
Operating Docker Swarm through the Docker Remote API
Using Docker Remote API to automate proxy configuration
Combining the Swarm listener with the proxy
What now?
8
Using Docker Stack and Compose YAML Files to Deploy Swarm Services
Using Docker Stack and Compose YAML Files to Deploy Swarm Services
Swarm cluster setup
Creating Swarm services through Docker stack commands
Deploying more stacks
To stack or not to stack
Cleanup
9
Defining Logging Strategy
Defining Logging Strategy
The need for centralized logging
Setting up ElasticSearch as the logging database
Setting up LogStash as the logs parser and forwarder
Forwarding logs from all containers running anywhere inside a Swarm cluster
Exploring logs
Discussing other logging solutions
What now?
10
Collecting Metrics and Monitoring the Cluster
Collecting Metrics and Monitoring the Cluster
The requirements of a cluster monitoring system
Choosing the right database to store system metrics
Creating the cluster
Prometheus metrics
Exporting system wide metrics
Scraping, querying, and visualizing Prometheus metrics
Using Grafana to create dashboards
Exploring Docker Swarm and container overview dashboard in Grafana
Adjusting services through dashboard metrics
Monitoring best practices
What now?
11
Embracing Destruction: Pets versus Cattle
Embracing Destruction: Pets versus Cattle
What now?
12
Creating and Managing a Docker Swarm Cluster in Amazon Web Services
Creating and Managing a Docker Swarm Cluster in Amazon Web Services
Installing AWS CLI and setting up the environment variables
Setting up a Swarm cluster with Docker Machine and AWS CLI
Setting up a Swarm cluster with Docker for AWS
Automatically setting up a Swarm cluster with Docker for AWS
Setting up a Swarm cluster with Packer and Terraform
Choosing the right tools to create and manage Swarm clusters in AWS
13
Creating and Managing a Docker Swarm Cluster in DigitalOcean
Creating and Managing a Docker Swarm Cluster in DigitalOcean
Setting up the environment variables
Setting up a Swarm cluster with Docker Machine and DigitalOcean API
Setting up a Swarm cluster with Packer and Terraform
Choosing the right tools to create and manage Swarm clusters in DigitalOcean
To DigitalOcean or not to DigitalOcean
14
Creating and Managing Stateful Services in a Swarm Cluster
Creating and Managing Stateful Services in a Swarm Cluster
Exploring the twelve-factor app methodology
Setting up a Swarm cluster and the proxy
Running stateful services without data persistence
Persisting stateful services on the host
Persisting stateful services on a Network File System
Data volume orchestration
Persisting stateful services with REX-Ray
Choosing the persistence method for stateful services
Adding REX-Ray to packer and terraform
Persisting stateful services without replication
Persisting stateful services with synchronization and replication
Persisting docker flow proxy state
Persisting MongoDB state
Initializing MongoDB replica set through a swarm service
What now?
15
Managing Secrets in Docker Swarm Clusters
Managing Secrets in Docker Swarm Clusters
Creating secrets
Consuming secrets
A real-world example of using secrets
Using secrets with Docker Compose
Common ways to use secrets
What now?
16
Monitor Your GitHub Repos with Docker and Prometheus
Monitor Your GitHub Repos with Docker and Prometheus
Docker, Prometheus, and Grafana
Getting started
Configuration
Post configuration
Install dashboard
Conclusion
About the author

Exploring logs

Having all the logs in a central database is a good start, but it does not allow us to explore them in an easy and user-friendly way. We cannot expect developers to start issuing requests to the ElasticSearch API whenever they want to explore what went wrong. We need a UI that allows us to visualize and filter logs. We need K from the ELK stack.

Note

A note to Windows users You might experience a problem with volumes not being mapped correctly with Docker Compose. If you may see an Invalid volume specification error, please export the environment variable COMPOSE_CONVERT_WINDOWS_PATHS set to 0:export COMPOSE_CONVERT_WINDOWS_PATHS=0 Please make sure that the variable is exported every time you run docker-compose or docker stack deploy.

Let's create one more service. This time, it'll be Kibana. Besides the need for this service to communicate with logspout and elasticsearch services, we want to expose it through the proxy, so we'll create swarm-listener and proxy services as well...

Previous Section
End of Section 6
Next Section