About the Book

Thanks to its extensive support for managing hundreds of containers that run cloud-native applications, Kubernetes is the most popular open source container orchestration platform that makes cluster management easy. This workshop adopts a practical approach to get you acquainted with the Kubernetes environment and its applications.

Starting with an introduction to the fundamentals of Kubernetes, you'll install and set up your Kubernetes environment. You'll understand how to write YAML files and deploy your first simple web application container using Pod. You'll then assign human-friendly names to Pods, explore various Kubernetes entities and functions, and discover when to use them. As you work through the chapters, this Kubernetes book will show you how you can make full-scale use of Kubernetes by applying a variety of techniques for designing components and deploying clusters. You'll also get to grips with security policies for limiting access to certain functions inside the cluster. Toward the end of the book, you'll get a rundown of Kubernetes advanced features for building your own controller and upgrading to a Kubernetes cluster without downtime.

By the end of this workshop, you'll be able to manage containers and run cloud-based applications efficiently using Kubernetes.

Audience

Whether you are new to the world of web programming or are an experienced developer or software engineer looking to use Kubernetes for managing and scaling containerized applications, you'll find this workshop useful. A basic understanding of Docker and containerization is necessary to make the most of this book.

About the Chapters

Chapter 1, Introduction to Kubernetes and Containers, begins with containerization technologies as well as various underlying Linux technologies that enable containerization. The chapter ends by introducing Kubernetes into the picture, while laying out the advantages it brings to the table.

Chapter 2, An Overview of Kubernetes, gives you your first hands-on introduction to Kubernetes and provides an overview of the architecture of Kubernetes.

Chapter 3, kubectl – Kubernetes Command Center, lays out the various ways of using kubectl while underlining the principle of declarative management.

Chapter 4, How to Communicate with Kubernetes (API Server), dives into the details of the Kubernetes API server and the various ways of communicating with it.

Chapter 5, Pods, introduces the basic Kubernetes object used to deploy any application.

Chapter 6, Labels and Annotations, covers the basic mechanism used in Kubernetes to group, classify, and link different objects.

Chapter 7, Kubernetes Controllers, introduces various Kubernetes controllers, such as Deployments and StatefulSets, among others, which are some of the key enablers of the declarative management approach.

Chapter 8, Service Discovery, describes how you can make different Kubernetes objects discoverable within the cluster as well as from outside the cluster.

Chapter 9, Storing and Reading Data on Disk, explains the various data storage abstractions offered by Kubernetes to enable applications to read and store data on disks.

Chapter 10, ConfigMaps and Secrets, teaches you how to decouple application configuration data from the application itself, while looking at the advantages of taking this approach.

Chapter 11, Build Your Own HA Cluster, walks you through setting up your own highly available, multi-node Kubernetes cluster on the Amazon Web Services (AWS) platform.

Chapter 12, Your Application and HA, lays out some concepts behind continuous integration using Kubernetes and demonstrates a few of them using a highly available, multi-node, managed Kubernetes cluster running on Amazon Elastic Kubernetes Service.

Chapter 13, Runtime and Network Security in Kubernetes, gives you an overview of the ways in which your application and cluster can be attacked, before covering the access control and security features offered by Kubernetes.

Chapter 14, Running Stateful Components in Kubernetes, teaches you how to properly use different Kubernetes abstractions to reliably deploy stateful applications.

Chapter 15, Monitoring and Autoscaling in Kubernetes, covers the ways in which you can monitor different Kubernetes objects and then use that information to scale the capacity of your cluster.

Chapter 16, Kubernetes Admission Controllers, describes how Kubernetes allows us to extend the functionalities provided by the API server to implement custom policies before a request is accepted by the API server.

Chapter 17, Advanced Scheduling in Kubernetes, describes how the scheduler places pods on the Kubernetes cluster. You will use advanced features to influence scheduler placement decisions for the pods.

Chapter 18, Upgrading Your Cluster without Downtime, teaches you how you can upgrade your Kubernetes platform to a newer version without suffering any downtime for your platform or application.

Chapter 19, Custom Resource Definitions in Kubernetes, shows you one of the main ways in which to extend the functionalities provided by Kubernetes. You will see how custom resources allow you to implement concepts specific to your own domain on your cluster.

Note

The solution to the activities presented in the chapters can be found at this address: https://packt.live/304PEoD.

Conventions

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, and user input are shown as follows: "Create a file named sample-pod.yaml in your current working directory."

A block of code, a terminal command, or text to create a YAML file is set as follows:

kubectl -n webhooks create secret tls webhook-server-tls \
--cert "tls.crt" \
--key "tls.key"

New important words are shown like this: "Kubernetes provides this capability via Admission Controllers."

Key parts of code snippets are highlighted as follows:

kind: Pod
metadata:
  name: infra-libraries-application-staging
  namespace: metadata-activity
  labels:
    environment: staging
    team: infra-libraries
  annotations:
      team-link: "https://jira-link/team-link-2"
spec:
  containers:

Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "On the left sidebar, click on Configuration and then on Data Sources."

Long code snippets are truncated and the corresponding names of the code files on GitHub are placed at the top of the truncated code. The permalinks to the entire code are placed below the code snippet. It should look as follows:

mutatingcontroller.go

46 //create the response with patch bytes 
47 var admissionResponse *v1beta1.AdmissionResponse 
48 admissionResponse = &v1beta1.AdmissionResponse { 
49     allowed: true, 
50     Patch:   patchBytes, 
51     PatchType: func() *v1beta1.PatchType { 
52         pt := v1beta1.PatchTypeJSONPatch 
53         return &pt 
54     }(), 
55 } 

The complete code for this example can be found at https://packt.live/35ieNiX.

Setting Up Your Environment

Before we explore the book in detail, we need to set up specific software and tools. In the following section, we shall see how to do that.

Hardware Requirements

You need at least a dual core CPU with virtualization support, 4 GB of memory, and 20 GB of free disk space.

Operating System Requirements

Our recommended operating system is Ubuntu 20.04 LTS or macOS 10.15. If you are using Windows, you can dual boot Ubuntu. We have provided the instructions for that at the end of this section.

Virtualization

You need to have virtualization features enabled on your hardware as well as your operating system.

In Linux, you can run the following command to check whether virtualization is enabled:

grep -E --color 'vmx|svm' /proc/cpuinfo

You should get a non-empty response to this command. If you get an empty response, then you don't have virtualization enabled.

In macOS, run the following command:

sysctl -a | grep -E --color 'machdep.cpu.features|VMX'

If virtualization is enabled, you should be able to see VMX in your output.

Note

You will not be able to follow the instructions in the book if your host environment is virtualized, since Minikube (by default) runs all Kubernetes components in a virtual machine, which will not work if the host environment itself is virtualized. It is possible to use Minikube without a hypervisor, but your results may sometimes be different compared to our demonstrations in this book. Therefore, our recommendation is that one of the recommended operating systems is directly installed on your machine.

Installation and Setup

This section lists installation instructions for all the software that you will need for this book. Since we are recommending Ubuntu, we will use the APT package manager to install most of the required software in Ubuntu.

For macOS, we recommend that you use Homebrew for convenience. You can install it by running this script in your terminal:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

The terminal output for this script will show you what changes will be applied and then ask for your confirmation. Once confirmed, the installation can be completed.

Updating Your Package Lists

Before you use APT to install any packages in Ubuntu, make sure that your package lists are up to date. Use the following command:

sudo apt update

Furthermore, you may choose to upgrade any upgradable packages on your machine by using the following command:

sudo apt upgrade

Similarly, in the case of macOS, update the package lists for Homebrew using the following command:

brew update

Installing Git

The code bundle for this workshop is available on our GitHub repository. You can use Git to clone the repository to get all the code files.

Use the following command to install Git on Ubuntu:

sudo apt install git-all

If you use Xcode on macOS, it is likely that you may already have Git installed. You can check that by running this command:

git --version

If you get a Command not found error, then you don't have it installed. You can install it via Homebrew using this command:

brew install git

jq

jq is a JSON parser that is useful for extracting any information from API responses in JSON format. You can install it using the following command on Ubuntu:

sudo apt install jq

You can use the following command for installation on macOS:

brew install jq

Tree

Tree is a package that will allow you to see the directory structure in the terminal. You can install it using the following command on Ubuntu:

sudo apt install tree

You can use the following command for installation on macOS:

brew install tree

The AWS CLI

The AWS command line tool is a CLI tool that you can use from your terminal to manage your AWS resources. You can install it using the installation instructions at this URL: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html.

Minikube and kubectl

Minikube allows us to create a single-node Kubernetes cluster for learning and testing purposes. kubectl is a command line interface tool that allows us to communicate with our cluster. You will find detailed installation instructions for these tools in Chapter 2, An Overview of Kubernetes.

Even if you have Minikube installed already, we recommend that you work with the version that is specified in Chapter 2, An Overview of Kubernetes, in order to guarantee the reproducibility of all instructions in this book.

Minikube requires you to have a hypervisor installed. We will go with VirtualBox.

VirtualBox

VirtualBox is an open source hypervisor that can be used by Minikube to virtualize a node for our cluster. Use the following command to install VirtualBox on Ubuntu:

sudo apt install virtualbox

For installation on macOS, first get the appropriate file from this link:

https://www.virtualbox.org/wiki/Downloads.

Then, follow the installation instructions mentioned here:

https://www.virtualbox.org/manual/ch02.html#installation-mac.

Docker

Docker is the default containerization engine used by Kubernetes. You will learn more about Docker in Chapter 1, Introduction to Kubernetes and Containers.

To install Docker, follow the installation instructions at this link:

https://docs.docker.com/engine/install/.

To install Docker in Mac, following the installation instructions at the following link:

https://docs.docker.com/docker-for-mac/install/.

To install Docker in Ubuntu, following the installation instructions at the following link:

https://docs.docker.com/engine/install/ubuntu/.

Go

Go is a programming language that is used to build the applications demonstrated in this book. Also, Kubernetes is written in Go. To install Go on your machine, use the following command for Ubuntu:

sudo apt install golang-go

For installation on macOS, use the following instructions:

1. Use the following command to install Go:

   brew install golang

   Note

   The code is tested with Go versions 1.13 and 1.14. Please make sure that you have these versions although the code is expected to work for all 1.x versions.

2. Now, we need to set a few environment variables. Use the following commands:

   mkdir - p $HOME/go
   export GOPATH=$HOME/go
   export GOROOT="$(brew --prefix golang)/libexec"
   export PATH="$PATH:${GOPATH}/bin:${GOROOT}/bin"

kops

kops is a command line interface tool that allows you to set up a Kubernetes cluster on AWS. The actual process of installing Kubernetes using kops is covered in Chapter 11, Build Your Own HA Cluster. To ensure the reproducibility of the instructions given in this book, we recommend that you install kops version 1.15.1.

For installation on Ubuntu, follow these steps:

1. Download the binary for kops version 1.15.1 using the following command:

   curl -LO https://github.com/kubernetes/kops/releases/download/1.15.0/kops-linux-amd64

2. Now, make the binary executable using the following command:

   chmod +x kops-linux-amd64

3. Add the executable to your path:

   sudo mv kops-linux-amd64 /usr/local/bin/kops

4. Check whether kops has been successfully installed by running the following command:

   kops version

   If kops has been successfully installed, you should get a response stating the version as 1.15.0.

For installation on macOS, follow these steps:

1. Download the binary for kops version 1.15.1 using the following command:

   curl -LO https://github.com/kubernetes/kops/releases/download/1.15.0/kops-darwin-amd64

2. Now, make the binary executable using the following command:

   chmod +x kops-darwin-amd64

3. Add the executable to your path:

   sudo mv kops-darwin-amd64 /usr/local/bin/kops

4. Check whether kops has been successfully installed by running the following command:

   kops version

   If kops has been successfully installed, you should get a response stating the version as 1.15.0.

Dual-Booting Ubuntu for Windows Users

In this section, you will find instructions on how to dual-boot Ubuntu if you are running Windows.

Note

Before installing any operating system, it is highly recommended that you back up your system state as well as all of your data.

Resizing Partitions

If you have Windows set up on your machine, it is most likely that your hard disk is completely utilized – that is, all of the available space is partitioned and formatted. We need to have some unallocated space on the hard disk. Hence, we will resize a partition with plenty of free space to make space for our Ubuntu partitions:

1. Open the Computer Management utility. Press Win + R and enter compmgmt.msc:
   

   Figure 0.1: The Computer Management utility on Windows

2. In the left side pane, go to the Storage > Disk Management option as shown here:
   

   Figure 0.2: Disk Management

   You will see a summary of all your partitions in the lower half of the screen. You can also see the drive letters associated with all of the partitions and information about the Windows boot drive. If you have a partition that has plenty of free space (20 GB +) and is neither the boot drive (C:), nor the recovery partition, nor the EFI system partition, this will be the ideal option to choose. If there's no such partition, then you can resize the C: drive.

3. In this example, we will choose the D: drive. You can right-click on any partition and open Properties to check the free space available:
   

   Figure 0.3: Checking the properties of the D: drive

   Now, before we resize the partition, we need to ensure that there are no errors on the filesystem or any hardware faults. We will do this by using the chkdsk utility on Windows.

4. Open Command Prompt by pressing Win + R and entering cmd.exe. Now, run the following command:

   chkdsk D: /f

   Replace the drive letter with the one that you want to use. You should see a response similar to the following:

   

   Figure 0.4: Scanning a drive for any filesystem errors

   Note that in this screenshot, Windows reports that it has scanned the filesystem and found no problems. If any problems are encountered for your case, you should get them fixed first to prevent the loss of data.

5. Now, come back to the Computer Management window, right-click on the desired drive, and then click on Shrink Volume, as shown here:
   

   Figure 0.5: Opening the Shrink Volume dialog box

6. In the prompt window, enter the amount of space you want to clear in the only field that you can edit. In this example, we are clearing approximately 25 GB of disk space by shrinking our D: drive:
   

   Figure 0.6: Clearing 25 GB by shrinking the existing volume

7. After you shrink your drive, you should be able to see unallocated space on your drive, as seen here:
   

Figure 0.7: Unallocated space after shrinking the volume

Now we are ready to install Ubuntu. But first, we need to download it and create a bootable USB, which is one of the most convenient installation media.

Creating a Bootable USB Drive to Install Ubuntu

You will need a flash drive with a minimum capacity of 4 GB. Note that all the data on this will be erased:

1. Download the ISO image for Ubuntu Desktop from this link: https://releases.ubuntu.com/20.04/.
2. Next, we need to burn the ISO image to a USB flash disk and create a bootable USB drive. There are many tools available for this, and you can use any of them. In this example, we are using Rufus, which is free and open source. You can get it from this link: https://www.fosshub.com/Rufus.html.
3. Once you have installed Rufus, plug in your USB flash disk and open Rufus. Ensure that the proper Device option is selected, as shown in the following screenshot.
4. Press the SELECT button under Boot selection and then open the Ubuntu 18.04 image that you have downloaded.
5. The choice for Partition scheme will depend on how your BIOS and your disk drive are configured. GPT will be the best option for most modern systems, while MBR will be compatible with older systems:
   

   Figure 0.8: Configurations for Rufus

6. You may leave all other options on default, and then press START. After completion, close Rufus. You now have a bootable USB drive ready to install Ubuntu.

Installing Ubuntu

Now, we will use the bootable USB drive to install Ubuntu:

1. To install Ubuntu, boot using the bootable installation media that we just created. In most cases, you should be able to do that by simply having the USB drive plugged in while starting up your machine. If you don't automatically boot into the Ubuntu setup, go into your BIOS settings and ensure that your USB device is at the highest boot priority and that Secure Boot is turned off. The instructions for entering the BIOS setup are usually displayed on the splash screen (the screen with your PC manufacturer logo when you start up your computer) that is displayed during POST checks. You may also have the option to enter a boot menu while starting up. Usually, you have to hold down Delete, F1, F2, F12, or some other key while your PC boots up. It depends on your motherboard's BIOS.

   You should see a screen with a Try Ubuntu or Install Ubuntu option. If you don't see this screen, and instead you see a shell with a message that begins with Minimal BASH Like Line Editing is Supported..., then it is likely that there may have been some data corruption while downloading the ISO file or creating your bootable USB drive. Check the integrity of the downloaded ISO file by calculating the MD5, SHA1, or SHA256 hash of your downloaded file and comparing it to the ones you can find in the files named MD5SUMS, SHA1SUMS, or SHA256SUMS on the Ubuntu download page mentioned earlier. Then, repeat the steps in the previous section to reformat and recreate the bootable USB drive.

   If you have set the highest boot priority to the correct USB device in the BIOS and you are still unable to boot using your USB device (your system may just ignore it and boot into Windows instead), then there are two most likely issues:

   - The USB drive was not properly configured to be recognized as a bootable device or the GRUB bootloader was not properly set up. Verifying the integrity of your downloaded image and recreating the bootable USB drive should fix this in most cases.

   - You have chosen the wrong Partition scheme option for your system configuration. Try the other one and recreate the USB drive.

2. Once you boot your machine using the USB drive, select Install Ubuntu.
3. Choose the language that you want and then press Continue.
4. On the next screen, choose the appropriate keyboard layout and continue to the next screen.
5. On the next screen, select Normal installation.

   Check the Download updates while installing Ubuntu and Install third-party software for graphics and Wi-Fi hardware and additional media formats options.

   Then, continue to the next screen.

6. On the next screen, select Install Ubuntu alongside Windows Boot Manager, and then click Install now. You will see a prompt describing the changes that Ubuntu will make to your system, such as the new partitions that will be created. Confirm the changes and proceed to the next screen.
7. On the next screen, choose your region and press Continue.
8. On the next screen, set your name (optional), username, computer name, and password, and then press Continue.

   The installation should now begin. It will take a while depending on your system configurations. Once the installation is complete, you will be prompted to restart your computer. Unplug your USB drive, and then click Restart Now.

   If you forget to remove your USB drive, you may boot back into the Ubuntu installation. In that case, just exit the setup. If a live instance of Ubuntu has been started up, restart your machine. Remember to remove the USB drive this time.

   If, after restarting, you boot directly into Windows with no option to choose the operating system, the likely issue is that the GRUB bootloader installed by Ubuntu has not taken precedence over the Windows bootloader. In some systems, the precedence/priority for bootloaders on your hard disk is set in the BIOS. You will need to explore your BIOS settings menu to find the appropriate setting. It may be named something similar to UEFI Hard Disk Drive Priorities. Ensure that GRUB/Ubuntu is set to the highest priority.

Other Requirements

Docker Hub account: You can create a free Docker account at this link: https://hub.docker.com/.

AWS account: You will need your own AWS account and some basic knowledge about using AWS. You can create an account here: https://aws.amazon.com/.

Note

The requirements of the exercises and activities in this book go beyond the AWS free tier, so you should be aware that you will incur bills for the use of the cloud service. You can use the pricing information available here: https://aws.amazon.com/pricing/.

Accessing the Code Files

You can find the complete code files of this book at https://packt.live/3bE3zWY.

After installing Git, you can clone the repository using the following command:

git clone https://github.com/PacktWorkshops/Kubernetes-Workshop
cd Kubernetes-Workshop

If you have any issues or questions about installation, please email us at [email protected].