Book Image

The Linux DevOps Handbook

By : Damian Wojsław, Grzegorz Adamowicz
3.5 (2)
Book Image

The Linux DevOps Handbook

3.5 (2)
By: Damian Wojsław, Grzegorz Adamowicz

Overview of this book

The Linux DevOps Handbook is a comprehensive resource that caters to both novice and experienced professionals, ensuring a strong foundation in Linux. This book will help you understand how Linux serves as a cornerstone of DevOps, offering the flexibility, stability, and scalability essential for modern software development and operations. You’ll begin by covering Linux distributions, intermediate Linux concepts, and shell scripting to get to grips with automating tasks and streamlining workflows. You’ll then progress to mastering essential day-to-day tools for DevOps tasks. As you learn networking in Linux, you’ll be equipped with connection establishment and troubleshooting skills. You’ll also learn how to use Git for collaboration and efficient code management. The book guides you through Docker concepts for optimizing your DevOps workflows and moves on to advanced DevOps practices, such as monitoring, tracing, and distributed logging. You’ll work with Terraform and GitHub to implement continuous integration (CI)/continuous deployment (CD) pipelines and employ Atlantis for automated software delivery. Additionally, you’ll identify common DevOps pitfalls and strategies to avoid them. By the end of this book, you’ll have built a solid foundation in Linux fundamentals, practical tools, and advanced practices, all contributing to your enhanced Linux skills and successful DevOps implementation.
Table of Contents (20 chapters)
1
Part 1: Linux Basics
6
Part 2: Your Day-to-Day DevOps Tools
12
Part 3: DevOps Cloud Toolkit

Introducing the distributions

After that bit of a lengthy but condensed history of the Linux OS, it is time to finally explore the few we have chosen to cover in this book. In this section, we will cover the factors we just listed, as we believe they are important in making a decision. Please remember though that while we strive to present you with objective facts and valuations, we cannot escape our own subjective views. Always evaluate on your own before you choose, as it’s highly possible that you will stick with this distribution for many years to come.

A point to note is that we won’t be covering distributions comprehensively. We will only try to create a foundation on which you, dear reader, must build through research.

Also, while you are learning, do not be afraid to hop from distribution to distribution. Only through real-life experiences will you fully understand which one covers your needs best.

Debian

Debian (https://www.debian.org/) is one of the oldest active Linux distributions. Its development is led by the community-supported Debian Project. It is known for two things – the sheer number of packages that the distribution provides and the slow release of stable versions. The latter has improved in recent years and stable releases are published every two years. Software is delivered in archives called packages. Debian packages’ names have a .deb file extension and are colloquially called debs. They are kept online in repositories and repositories are broken down into pools. Repositories offer almost 60,000 packages with software in the latest stable release.

Debian always has three versions available (so-called branches) – stable, testing, and unstable. The releases are named after characters from the Toy Story movie franchise. The latest stable release – version 11 – is called Bullseye.

The unstable branch is the rolling branch for developers, people who like living on the edge, or those who require the newest software more than they require stability. Software is accepted into the unstable branch with minimal testing.

The testing branch is where, as the name implies, the testing happens. A lot of testing happens here, thanks to the end users. Packages come here from the unstable branch. The software here is still newer than in the stable branch but not as fresh as in the unstable branch. A few months before the new stable release, the testing branch is frozen. It means that no new software will be accepted, and new versions of the already accepted packages are allowed only if they fix bugs.

After a few months, testing becomes the stable branch. The software is updated only for security fixes.

This distribution is available for many hardware platforms – Intel, ARM, PowerPC, and so on. Along with unofficial ports, there are a multitude of hardware platforms on which you can install it.

Debian is viewed as the most stable distribution there is, and it is used as a platform for various compute clusters, so it is generally installed on bare-metal servers somewhere in a rack in a data center and intended for use consistently over many years.

According to W3Techs (https://w3techs.com/technologies/details/os-linux), Debian makes up for 16% of all servers running on the internet. Its derivative, Ubuntu, runs 33% of them. Together, they account for 49% of all servers. This makes administration skills related to Debian highly marketable.

Ubuntu Linux

The Ubuntu Linux distribution (https://ubuntu.com/) is widely credited for making Linux popular on personal computers, and rightly so. Sponsored by Canonical, its mission was to make Linux easily usable for most people. It was one of the first, if not the first, Linux versions to distribute non-free and non-open binary drivers and libraries that made desktop use simpler and more comfortable.

Famously, the first bug report opened for Ubuntu distribution by Mark Shuttleworth (Canonical and Ubuntu founder) was, “Microsoft has majority market share.

The distribution itself is based on Debian Linux, and in the beginning, being fully binary-compatible was one of the major objectives. As the development has progressed, this has lost some of its importance.

This distribution is developed by the community and Canonical. The main source of income for the company is premium services related to Ubuntu Linux – support, training, and consultations.

Due to the very close-knit relationship between Debian Linux and Ubuntu Linux, many developers and maintainers for one distribution serve the same roles in the other one. This results in a lot of software being packaged for both distributions in parallel.

Ubuntu has three major flavors – Desktop, Server, and Core (for the internet of things). Desktop and Server may differ slightly in how services are configured out of the box, and Core differs a lot

The software is distributed in .deb packages, the same as with Debian, and the sources are actually imported from the Debian unstable branch. However, this doesn’t mean you can install Debian packages on Ubuntu or vice versa, as they are not necessarily binary-compatible. It should be possible to rebuild and install your own version.

There are four package repositories per release – the free and non-free software supported officially by Canonical is called main and restricted, respectively. Free and non-free software delivered and maintained by the community is called universe and multiverse, respectively.

Important note

A word of advice – a widely accepted practice of system upgrades between major versions is to wait for the first sub-release. So, if the currently installed version of the distribution is 2.5 and the new version 3.0 is released, it is wise to wait until 3.1 or even 3.2 is released and upgrade then. This is applicable to all the distributions we list here.

The Long-Term Support (LTS) versions are supported for five years. A new LTS version is released every two years. It is possible to negotiate extended support. This gives a very good timeline to plan major upgrades. A new Ubuntu version is released every six months.

Ubuntu Linux is widely adopted in education and government projects. Famously, the city of Munich, between 2004 and 2013, migrated over 14,000 municipal desktop computers to a variant of Ubuntu with the KDE desktop environment. While the migration saw disturbances politically – other operating system vendors lobbied strongly against this migration – it was considered a success technically.

Ubuntu is the Linux of choice for personal computers. Canonical works very closely with hardware vendors, notably Lenovo and Dell, but lately also with HP, to ensure full compatibility between the distribution and the computers. Dell sells its flagship laptops with Ubuntu preinstalled.

Several sources cite Ubuntu Linux as the most installed Linux distribution on servers and personal computers. The actual number can only be estimated, as Ubuntu doesn’t require any subscription or registration.

As a byproduct of Ubuntu Linux’s popularity, software vendors, more often than not, offer .deb packages of their software, if they release a Linux version. This is especially true for desktop software.

The amount of unofficial versions, clones, or modified distributions based on Ubuntu is staggering.

Ubuntu has a very active community, both organized and unorganized. It’s quite easy to get a hold of a group of users near your city. This also directly translates to the amount of tutorials and documentation on the internet.

Ubuntu Linux, especially under a support plan, is installed as a foundation for many cloud computing infrastructure deployments. Many telecoms, banking, and insurance companies have chosen Ubuntu Server as their foundation.

Red Hat Enterprise Linux (RHEL)

RHEL (https://www.redhat.com/en/technologies/linux-platforms/enterprise-linux) is a spiritual successor of Red Hat Linux and is developed and maintained by Red Hat Inc. (https://www.redhat.com/). Its main target is the commercial entities market. It is possible to use RHEL for free for development or in production with up to 16 servers (at the time of writing). However, the main advantage of this distribution is the enormous pool of articles that help solve issues and the assistance of support engineers, although the latter can only be acquired through a paid support plan.

RHEL is considered a very stable and solid distribution. It is one of the main choices for banks, insurance companies, and financial markets. It lacks many popular desktop software packages, but on the server side of things, especially as an OS to run other commercial applications, it is a first-class citizen.

The software is distributed in online repository packages that end with .rpm, hence the name RPMs. The main tool to administer the packages is RPM, with more sophisticated tools – yum, and lately its successor, dnfalso available.

In the true spirit of an open source-based company, Red Hat makes sources for its distribution available. This has led to the creation of a famous free and open clone of RHEL – CentOS. Until fairly recently, it had been quite a popular choice for people who wanted to use RHEL but didn’t want to, or couldn’t, pay a subscription. In 2014, CentOS joined the Red Hat company, and in 2020, Red Hat announced that the versioned releases of CentOS would no longer be available; there would only be the so-called rolling release, which constantly updates packages and does not mirror the RHEL releases. This resulted in a very heated reaction from CentOS users. The original CentOS founder, Gregory Kurtzer, started another clone of RHEL called Rocky Linux. Its main objective is the same as the original CentOS – to deliver a free, open, and community-driven distribution, fully binary-compatible with RHEL.

The RHEL distribution delivers stable versions every few years and supports them for 10 years, starting from release 5. The full support, however, is offered only for a few years. For the rest of the time, Red Hat provides only security fixes and critical updates for your systems, with no new package versions being introduced. Still, this life cycle is what users with large installations or mission-critical systems came to like.

As with Ubuntu, it is possible to negotiate extended support time.

The Red Hat company has a turbulent relationship with the open source community. While the company mostly plays fair, there have been some decisions that the community didn’t like. Lately, it was Red Hat’s decision to change the CentOS release model to a rolling release (https://lists.centos.org/pipermail/centos-announce/2020-December/048208.html).

RHEL, like Ubuntu, is the chosen foundation for commercially supported deployments of cloud infrastructure.

Fedora Linux

Fedora (https://fedoraproject.org/wiki/Fedora_Project_Wiki) is a distribution associated with the Red Hat company. While more than 50% of its developers and maintainers are community members not affiliated with Red Hat, the company holds full stewardship over the development. It is a RHEL upstream, which means that this is the real development frontend for the actual RHEL. It doesn’t mean that everything from Fedora is included in the release of RHEL. However, following Fedora closely will yield insight into the current direction of the RHEL distribution.

Contrary to RHEL, for which Fedora is the foundation, the new releases happen every six months. It uses the same package type as RHEL, RPM.

Fedora is considered a fast-paced distribution. It quickly adopts the newest and bleeding-edge versions of packages.

CentOS

CentOS (https://centos.org) used to be the go-to free version of RHEL. The name is an acronym for Community Enterprise Operating System. Its main goal was to be fully binary-compatible with RHEL and adhere to the same releases and numbering scheme. In 2014, CentOS joined Red Hat, but it was promised that the distribution would keep its independence from the company while benefiting from development and testing resources. Unfortunately, in 2020, Red Hat announced that CentOS 8 would be the last numbered release, and from then on, CentOS Stream would be the only variant. CentOS Stream is a midstream version. This means it sits in the middle between bleeding-edge and fast-paced Fedora and stable and production-ready RHEL. The difference between CentOS Stream and CentOS is that Stream is a development variant, while CentOS was simply a rebuilt and repackaged mirror of the actual final product, RHEL.

All the knowledge, skills, and experience gained when working with RHEL are 100% applicable to CentOS. Since CentOS is the third most-deployed Linux distribution on servers, according to W3Techs (https://w3techs.com/technologies/details/os-linux), the skills are very marketable.

Rocky Linux

As a response to the situation with the CentOS distribution, its founder announced the creation of Rocky Linux (https://rockylinux.org/). The goals are the same as the original CentOS. The release scheme and numbering follow RHEL. Shortly after the announcement, the GitHub repository of Rocky Linux became top trending (https://web.archive.org/web/20201212220049/https://github.com/trending). Rocky Linux is 100% binary-compatible with CentOS. The project has released a set of tools that easily migrate from CentOS to Rocky Linux without reinstalling the system.

The distribution is quite young, having been founded in 2020, and its popularity is still to be determined. It has made a lot of noise in the community, and it seems that a steady stream of CentOS users have moved to Rocky Linux as their preferred choice.

A very important contribution to the open source world from the Rocky Linux project is the build system. It ensures that even if Rocky Linux shuts down, the community will be able to easily start up a new RHEL clone.

All the skills, knowledge, and articles for RHEL and CentOS are 100% applicable to Rocky Linux. All the software that runs on RHEL and CentOS should run without any modifications on Rocky Linux too.

Alpine

Alpine Linux (https://alpinelinux.org/) is an interesting one. The main programming library and most basic command-line tools are not from the GNU Project. Also, the services management system, currently systemd in most distributions, is uncommon. This makes some of the administration skills from other major distributions non-applicable. The strength of Alpine lies in its size (which is rather small), its security-first mindset, and one of the fastest boot times among existing Linux distributions. Those characteristics, with the boot time being admittedly more important, make it the most popular choice for containers. If you run containerized software or build your own container images, it is very likely that it is on Alpine Linux.

Alpine has its roots in the LEAF (Linux Embedded Appliance Framework; see: https://bering-uclibc.zetam.org/wiki/Main_Page) project – a Linux distribution that fits on a single floppy disk. LEAF is currently a popular choice for embedded markets, routers, and firewalls. Alpine is a bigger distribution, but that sacrifice had to be made, since developers wanted to include several useful but rather large software packages.

The package manager is called apk. The build system is borrowed from another distribution called Gentoo Linux. As Gentoo builds software as it installs it, the portage obviously contains a lot of logic around building software that is used as a part of an OS.

Alpine can run from RAM entirely. There’s even a special mechanism that allows you to initially only load a few required packages from the boot device, and it can be achieved using Alpine’s Local Backup Utility (LBU).

As mentioned before, this is a preferred distribution for container images. You won’t see it running on a large server installation often, if at all. When we cross over to the cloud world, chances are you’ll see a lot of Alpine Linux.

Having said that, every single one of those distributions has a variant for the cloud as a container base image – a way to run your software in the true cloud way.

In this chapter, you learned the basics of popular Linux distributions and how they are different from one another. You should now have some understanding of what you can choose from and what consequences you will need to face – good and bad. To give you an even better idea of how to interact with some cherry-picked Linux distributions, we will look at how to interact with a system using your keyboard in Chapter 2.