Book Image

Linux Shell Scripting Cookbook - Third Edition

By : Clif Flynt, Sarath Lakshman, Shantanu Tushar
Book Image

Linux Shell Scripting Cookbook - Third Edition

By: Clif Flynt, Sarath Lakshman, Shantanu Tushar

Overview of this book

The shell is the most powerful tool your computer provides. Despite having it at their fingertips, many users are unaware of how much the shell can accomplish. Using the shell, you can generate databases and web pages from sets of files, automate monotonous admin tasks such as system backups, monitor your system's health and activity, identify network bottlenecks and system resource hogs, and more. This book will show you how to do all this and much more. This book, now in its third edition, describes the exciting new features in the newest Linux distributions to help you accomplish more than you imagine. It shows how to use simple commands to automate complex tasks, automate web interactions, download videos, set up containers and cloud servers, and even get free SSL certificates. Starting with the basics of the shell, you will learn simple commands and how to apply them to real-world issues. From there, you'll learn text processing, web interactions, network and system monitoring, and system tuning. Software engineers will learn how to examine system applications, how to use modern software management tools such as git and fossil for their own work, and how to submit patches to open-source projects. Finally, you'll learn how to set up Linux Containers and Virtual machines and even run your own Cloud server with a free SSL Certificate from letsencrypt.org.
Table of Contents (14 chapters)

Displaying output in a terminal

Users interact with the shell environment via a terminal session. If you are running a GUI-based system, this will be a terminal window. If you are running with no GUI, (a production server or ssh session), you will see the shell prompt as soon as you log in.

Displaying text in the terminal is a task most scripts and utilities need to perform regularly. The shell supports several methods and different formats for displaying text.

Getting ready

Commands are typed and executed in a terminal session. When a terminal is opened, a prompt is displayed. The prompt can be configured in many ways, but frequently resembles this:

username@hostname$

Alternatively, it can also be configured as root@hostname # or simply as $ or #.

The $ character represents regular users and # represents the administrative user root. Root is the most privileged user in a Linux system.

It is a bad idea to directly use the shell as the root user (administrator) to perform tasks. Typing errors have the potential to do more damage when your shell has more privileges. It is recommended that you log in as a regular user (your shell may denote this as $ in the prompt), and use tools such as sudo to run privileged commands. Running a command as sudo <command> <arguments> will run it as root.

A shell script typically begins with a shebang:

#!/bin/bash

Shebang is a line on which #! is prefixed to the interpreter path. /bin/bash is the interpreter command path for Bash. A line starting with a # symbol is treated by the bash interpreter as a comment. Only the first line of a script can have a shebang to define the interpreter to be used to evaluate the script.

A script can be executed in two ways:

  1. Pass the name of the script as a command-line argument:
        bash myScript.sh
  1. Set the execution permission on a script file to make it executable:
        chmod 755 myScript.sh
        ./myScript.sh.

If a script is run as a command-line argument for bash, the shebang is not required. The shebang facilitates running the script on its own. Executable scripts use the interpreter path that follows the shebang to interpret a script.

Scripts are made executable with the chmod command:

$ chmod a+x sample.sh

This command makes a script executable by all users. The script can be executed as follows:

$ ./sample.sh #./ represents the current directory

Alternatively, the script can be executed like this:

$ /home/path/sample.sh # Full path of the script is used

The kernel will read the first line and see that the shebang is #!/bin/bash. It will identify /bin/bash and execute the script as follows:

$ /bin/bash sample.sh

When an interactive shell starts, it executes a set of commands to initialize settings, such as the prompt text, colors, and so on. These commands are read from a shell script at ~/.bashrc (or ~/.bash_profile for login shells), located in the home directory of the user. The Bash shell maintains a history of commands run by the user in the ~/.bash_history file.

The ~ symbol denotes your home directory, which is usually /home/user, where user is your username or /root for the root user. A login shell is created when you log in to a machine. However, terminal sessions you create while logged in to a graphical environment (such as GNOME, KDE, and so on), are not login shells. Logging in with a display manager such as GDM or KDM may not read a .profile or .bash_profile (most don't), but logging in to a remote system with ssh will read the .profile. The shell delimits each command or command sequence with a semicolon or a new line. Consider this example: $ cmd1 ; cmd2
This is equivalent to these: 
$ cmd1
$ cmd2

A comment starts with # and proceeds up to the end of the line. The comment lines are most often used to describe the code, or to disable execution of a line of code during debugging:

# sample.sh - echoes "hello world"
echo "hello world"

Now let's move on to the basic recipes in this chapter.

How to do it...

The echo command is the simplest command for printing in the terminal.

By default, echo adds a newline at the end of every echo invocation:

$ echo "Welcome to Bash"
Welcome to Bash

Simply, using double-quoted text with the echo command prints the text in the terminal. Similarly, text without double quotes also gives the same output:

$ echo Welcome to Bash
Welcome to Bash

Another way to do the same task is with single quotes:

$ echo 'text in quotes'

These methods appear similar, but each has a specific purpose and side effects. Double quotes allow the shell to interpret special characters within the string. Single quotes disable this interpretation.

Consider the following command:

$ echo "cannot include exclamation - ! within double quotes"

This returns the following output:

bash: !: event not found error

If you need to print special characters such as !, you must either not use any quotes, use single quotes, or escape the special characters with a backslash (\):

$ echo Hello world !

Alternatively, use this:

$ echo 'Hello world !'

Alternatively, it can be used like this:

$ echo "Hello World\!" #Escape character \ prefixed.

When using echo without quotes, we cannot use a semicolon, as a semicolon is the delimiter between commands in the Bash shell:

echo hello; hello 

From the preceding line, Bash takes echo hello as one command and the second hello as the second command.

Variable substitution, which is discussed in the next recipe, will not work within single quotes.

Another command for printing in the terminal is printf. It uses the same arguments as the C library printf function. Consider this example:

$ printf "Hello world"

The printf command takes quoted text or arguments delimited by spaces. It supports formatted strings. The format string specifies string width, left or right alignment, and so on. By default, printf does not append a newline. We have to specify a newline when required, as shown in the following script:

#!/bin/bash
#Filename: printf.sh

printf  "%-5s %-10s %-4s\n" No Name  Mark
printf  "%-5s %-10s %-4.2f\n" 1 Sarath 80.3456
printf  "%-5s %-10s %-4.2f\n" 2 James 90.9989
printf  "%-5s %-10s %-4.2f\n" 3 Jeff 77.564

We will receive the following formatted output:

No    Name       Mark
1     Sarath     80.35
2     James      91.00
3     Jeff       77.56

How it works...

The %s, %c, %d, and %f characters are format substitution characters, which define how the following argument will be printed. The %-5s string defines a string substitution with left alignment (- represents left alignment) and a 5 character width. If - was not specified, the string would have been aligned to the right. The width specifies the number of characters reserved for the string. For Name, the width reserved is 10. Hence, any name will reside within the 10-character width reserved for it and the rest of the line will be filled with spaces up to 10 characters total.

For floating point numbers, we can pass additional parameters to round off the decimal places.

For the Mark section, we have formatted the string as %-4.2f, where .2 specifies rounding off to two decimal places. Note that for every line of the format string, a newline (\n) is issued.

There's more...

While using flags for echo and printf, place the flags before any strings in the command, otherwise Bash will consider the flags as another string.

Escaping newline in echo

By default, echo appends a newline to the end of its output text. Disable the newline with the -n flag. The echo command accepts escape sequences in double-quoted strings as an argument. When using escape sequences, use echo as echo -e "string containing escape sequences". Consider the following example:

echo -e "1\t2\t3"
1  2  3

Printing a colored output

A script can use escape sequences to produce colored text on the terminal.

Colors for text are represented by color codes, including, reset = 0, black = 30, red = 31, green = 32, yellow = 33, blue = 34, magenta = 35, cyan = 36, and white = 37.

To print colored text, enter the following command:

echo -e "\e[1;31m This is red text \e[0m"

Here, \e[1;31m is the escape string to set the color to red and \e[0m resets the color back. Replace 31 with the required color code.

For a colored background, reset = 0, black = 40, red = 41, green = 42, yellow = 43, blue = 44, magenta = 45, cyan = 46, and white=47, are the commonly used color codes.

To print a colored background, enter the following command:

echo -e "\e[1;42m Green Background \e[0m"

These examples cover a subset of escape sequences. The documentation can be viewed with man console_codes.