Book Image

The DevOps 2.5 Toolkit

By : Viktor Farcic

Book Image

The DevOps 2.5 Toolkit

By: Viktor Farcic

Overview of this book

Building on The DevOps 2.3 Toolkit: Kubernetes, and The DevOps 2.4 Toolkit: Continuous Deployment to Kubernetes, Viktor Farcic brings his latest exploration of the Docker technology as he records his journey to monitoring, logging, and autoscaling Kubernetes. The DevOps 2.5 Toolkit: Monitoring, Logging, and Auto-Scaling Kubernetes: Making Resilient, Self-Adaptive, And Autonomous Kubernetes Clusters is the latest book in Viktor Farcic’s series that helps you build a full DevOps Toolkit. This book helps readers develop the necessary skillsets needed to be able to operate Kubernetes clusters, with a focus on metrics gathering and alerting with the goal of making clusters and applications inside them autonomous through self-healing and self-adaptation. Work with Viktor and dive into the creation of self-adaptive and self-healing systems within Kubernetes.

Free Chapter

Autoscaling Deployments and StatefulSets Based on Resource Usage

Autoscaling Deployments and StatefulSets Based on Resource Usage

Creating a cluster

Observing Metrics Server data

Auto-scaling Pods based on resource utilization

To replicas or not to replicas in Deployments and StatefulSets?

Auto-scaling Nodes of a Kubernetes Cluster

Auto-scaling Nodes of a Kubernetes Cluster

Creating a cluster

Setting up Cluster Autoscaling

Scaling up the cluster

The rules governing nodes scale-up

Scaling down the cluster

The rules governing nodes scale-down

Can we scale up too much or de-scale to zero nodes?

Cluster Autoscaler compared in GKE, EKS, and AKS

Collecting and Querying Metrics and Sending Alerts

Collecting and Querying Metrics and Sending Alerts

Creating a cluster

Choosing the tools for storing and querying metrics and alerting

A quick introduction to Prometheus and Alertmanager

Which metric types should we use?

Alerting on latency-related issues

Alerting on traffic-related issues

Alerting on error-related issues

Alerting on saturation-related issues

Alerting on unschedulable or failed pods

Upgrading old Pods

Measuring containers memory and CPU usage

Comparing actual resource usage with defined requests

Comparing actual resource usage with defined limits

Debugging Issues Discovered Through Metrics and Alerts

Debugging Issues Discovered Through Metrics and Alerts

Creating a cluster

Facing a disaster

Using instrumentation to provide more detailed metrics

Using internal metrics to debug potential issues

Extending HorizontalPodAutoscaler with Custom Metrics

Extending HorizontalPodAutoscaler with Custom Metrics

Creating a cluster

Using HorizontalPodAutoscaler without metrics adapter

Exploring Prometheus Adapter

Creating HorizontalPodAutoscaler with custom metrics

Combining Metric Server data with custom metrics

The complete HorizontalPodAutoscaler flow of events

Reaching nirvana

Visualizing Metrics and Alerts

Visualizing Metrics and Alerts

Creating a cluster

Which tools should we use for dashboards?

Installing and setting up Grafana

Importing and customizing pre-made dashboards

Creating custom dashboards

Creating semaphore dashboards

A better dashboard for big screens

Prometheus alerts vs. Grafana notifications vs. semaphores vs. graph alerts

Collecting and Querying Logs

Collecting and Querying Logs

Creating a cluster

Exploring logs through kubectl

Choosing a centralized logging solution

Exploring logs collection and shipping

Exploring centralized logging through Papertrail

Combining GCP Stackdriver with a GKE cluster

Combining AWS CloudWatch with an EKS cluster

Combining Azure Log Analytics with an AKS cluster

Exploring centralized logging through Elasticsearch, Fluentd, and Kibana

Switching to Elasticsearch for storing metrics

What should we expect from centralized logging?

What Did We Do?

What Did We Do?

Other Books You May Enjoy

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Creating a cluster

We'll continue using definitions from the vfarcic/k8s-specs (https://github.com/vfarcic/k8s-specs) repository. To be on the safe side, we'll pull the latest version first.

All the commands from this chapter are available in the 02-ca.sh (https://gist.github.com/vfarcic/a6b2a5132aad6ca05b8ff5033c61a88f) Gist.

 1  cd k8s-specs
 2
 3  git pull

Next, we need a cluster. Please use the Gists below as inspiration to create a new cluster or to validate that the one you already fulfills all the requirements.

A note to AKS users
At the time of this writing (October 2018), Cluster Autoscaler does not (always) work in Azure Kubernetes Service (AKS). Please jump to Setting up Cluster Autoscaler in AKS section for more info and the link to instructions how to set it up.

gke-scale.sh: GKE with 3 n1-standard-1 worker nodes, with tiller, and with the --enable-autoscaling...