Book Image

Linux Shell Scripting Cookbook - Third Edition

By : Clif Flynt, Sarath Lakshman, Shantanu Tushar
Book Image

Linux Shell Scripting Cookbook - Third Edition

By: Clif Flynt, Sarath Lakshman, Shantanu Tushar

Overview of this book

The shell is the most powerful tool your computer provides. Despite having it at their fingertips, many users are unaware of how much the shell can accomplish. Using the shell, you can generate databases and web pages from sets of files, automate monotonous admin tasks such as system backups, monitor your system's health and activity, identify network bottlenecks and system resource hogs, and more. This book will show you how to do all this and much more. This book, now in its third edition, describes the exciting new features in the newest Linux distributions to help you accomplish more than you imagine. It shows how to use simple commands to automate complex tasks, automate web interactions, download videos, set up containers and cloud servers, and even get free SSL certificates. Starting with the basics of the shell, you will learn simple commands and how to apply them to real-world issues. From there, you'll learn text processing, web interactions, network and system monitoring, and system tuning. Software engineers will learn how to examine system applications, how to use modern software management tools such as git and fossil for their own work, and how to submit patches to open-source projects. Finally, you'll learn how to set up Linux Containers and Virtual machines and even run your own Cloud server with a free SSL Certificate from letsencrypt.org.
Table of Contents (14 chapters)

Sending output from one command to another

One of the best features of the Unix shells is the ease of combining many commands to produce a report. The output of one command can appear as the input to another, which passes its output to another command, and so on. The output of this sequence can be assigned to a variable. This recipe illustrates how to combine multiple commands and how the output can be read.

Getting ready

The input is usually fed into a command through stdin or arguments. The output is sent to stdout or stderr. When we combine multiple commands, we usually supply input via stdin and generate output to stdout.

In this context, the commands are called filters. We connect each filter using pipes, sympolized by the piping operator (|), like this:

$ cmd1 | cmd2 | cmd3 

Here, we combine three commands. The output of cmd1 goes to cmd2, the output of cmd2 goes to cmd3, and the final output (which comes out of cmd3) will be displayed on the monitor, or directed to a file.

How to do it...

Pipes can be used with the subshell method for combining outputs of multiple commands.

  1. Let's start with combining two commands:
        $ ls | cat -n > out.txt

The output of ls (the listing of the current directory) is passed to cat -n, which in turn prepends line numbers to the input received through stdin. The output is redirected to out.txt.

  1. Assign the output of a sequence of commands to a variable:
        cmd_output=$(COMMANDS)

This is called the subshell method. Consider this example:

        cmd_output=$(ls | cat -n)
        echo $cmd_output

Another method, called back quotes (some people also refer to it as back tick) can also be used to store the command output:

        cmd_output=`COMMANDS`

Consider this example:

        cmd_output=`ls | cat -n`
        echo $cmd_output

Back quote is different from the single-quote character. It is the character on the ~ button on the keyboard.

There's more...

There are multiple ways of grouping commands.

Spawning a separate process with subshell

Subshells are separate processes. A subshell is defined using the ( ) operators:

  • The pwd command prints the path of the working directory
  • The cd command changes the current directory to the given directory path:
        $> pwd 
        / 
        $> (cd /bin; ls) 
        awk bash cat... 
        $> pwd 
        /

When commands are executed in a subshell, none of the changes occur in the current shell; changes are restricted to the subshell. For example, when the current directory in a subshell is changed using the cd command, the directory change is not reflected in the main shell environment.

Subshell quoting to preserve spacing and the newline character

Suppose we are assigning the output of a command to a variable using a subshell or the back quotes method, we must use double quotes to preserve the spacing and the newline character (\n). Consider this example:

$ cat text.txt
1
2
3

$ out=$(cat text.txt)
$ echo $out
1 2 3 # Lost \n spacing in 1,2,3

$ out="$(cat text.txt)"
$ echo $out
1
2
3