Linux Shell Scripting Essentials

We can print the output of commands in console in an interactive way. Console is also known as a standard input and output stream. To print anything in a bash console, use the echo command followed by what is to be printed:

$ echo Hello World
Hello World

Alternatively, put the text to be printed in double quotes:

$  echo "Hello World"
Hello World

You can also put the text to be printed in single quotes:

$ echo 'Hello World'
Hello World

We can also use the printf command in shell programming for printing. The printf command also supports formatted printing, similar to what we have in C programming language— the printf( ) function:

$ printf "Hello World"
Hello World$

Here, after the output, we see the command prompt ($) because printf doesn't add a default newline after execution while echo does. So, we have to explicitly add the newline (\n) in the printf statement to add a newline:

$ printf "Hello World\n"
Hello World

Similar to the C printf( ), we can specify formatted printing in bash. The syntax of bash printf is as follows:

printf FORMAT [ARGUMENTS]

FORMAT is a string that describes the format specifications and is specified within double quotes. ARGUMENTS can be the value or a variable corresponding to format specification. Format specification consists of the percentage (%) sign followed by format specifier. Format specifiers are explained in the following table:

The following examples demonstrate how to use format specification for printing different data type format in shell:

$ printf "%d mul %f = %f\n" 6 6.0 36.0
6 mul 6.000000 = 36.000000
$ printf "%s Scripting\n" Shell
Shell Scripting

We can also optionally specify a modifier in format specification to align an output to provide better formatting to the output. Format modifiers are placed between % and the format specifier character. The following table explains format modifiers:

The following example demonstrates how to use format modifiers to improve printing formatting:

$ printf "%d mul %.2f = %.2f\n" 6 6.0 36.0
6 mul 6.00 = 36.00

Interactive printing is good if we have to print one or two lines, but for a lot of printing, it's good and preferred to write a script file. A script file will contain all the instructions and we can run a script file to perform the needed task.

Now, we are going to create a bash script file that makes use of the echo and printf commands and print messages:

#!/bin/bash
#Filename: print.sh
#Description: print and echo

echo "Basic mathematics"
printf "%-7d %-7s %-7.2f =\t%-7.2f\n" 23 plus 5.5 28.5
printf "%-7.2f %-7s %-7d =\t%-7.2f\n" 50.50 minus 20 30.50 
printf "%-7d %-7s %-7d =\t%-7d\n" 10 mul 5 50
printf "%-7d %-7s %-7d =\t%-7.2f\n" 27 div 4 6.75

The first line in bash script represents the path of the interpreter used. The second line is a comment line telling the filename of a script file. In shell script, we use # to add a comment. Furthermore, the echo command will print strings written within double quotes. For the rest, we have used printf to print formatted output.

To run this script, we will first provide execute permission to a user/owner of this script:

$ chmod u+x print.sh

Then, run the script file in console as follows:

$ ./print.sh

The result after running this script will look as follows:

A variable name can be a combination of alphanumeric and underscore. Also, the name of the variable can't start with a number. The variable names in shell script are case-sensitive. Special characters, such as *, -, +, ~, ., ^, and so on, are not used in variable names because they have a special meaning in shell. The following table illustrates the correct and incorrect ways of naming a variable:

We can assign a value to a variable by using an assignment (=) operator and followed by a value. While assigning a variable value, there shouldn't be any space before and after the assignment operator. Also, a variable can't be declared alone; it has to be followed by its initial value assignment:

$ book="Linux Shell Scripting"  # Stores string value

$ book = "Linux Shell Scripting"  # Wrong, spaces around = operator
$ total_chapters=8    # Stores integer value
$ number_of_pages=210    # Stores integer value
$ average_pages_per_chapter=26.25    # Stores float value

So, it's quite easy to declare and assign a value to a variable in shell script. You don't have to worry about the data type of a variable on the left-hand side. Whatever value you provide on the right-hand side, the variable stores that value.

To access a variable value, use a dollar sign ($) operator followed by a variable name:

#!/bin/bash
#Filename: variables.sh
#Description: Basic variable definition and accessing them

book="Linux Shell Scripting"
total_chapters=8
number_of_pages=210
average_pages_per_chapter=26.25

echo "Book name - $book"
echo "Number of Chapters - $total_chapters"
printf "Total number of pages in book - $number_of_pages\n"
printf "Average pages in each chapter - %-.2f\n" $average_pages_per_chapter

The result of this script will look as follows:

Book name - Linux Shell Scripting
Number of Chapters - 8
Total number of pages in book - 210
Average pages in each chapter – 26.25

We can remove the value of a variable using the unset keyword in bash. Using unset to a variable deletes and resets it to null:

#!/bin/bash
#Filename: unset.sh
#Description: removing value of a variable

fruit="Apple"
quantity=6
echo "Fruit = $fruit , Quantity = $quantity"
unset fruit
echo "Fruit = $fruit , Quantity = $quantity"

The result after running this script will look as follows:

Fruit = Apple , Quantity = 6
Fruit =  , Quantity = 6

It's clear that we used unset on a fruit variable, so when we try to access a variable fruit after unsetting it in line no. 8, it prints nothing. The quantity variable still retains its value because we haven't used unset on it.

We can also create the constant variable in bash whose value can't be changed. The readonly keyword is used to declare a constant variable. We can also use declare -r followed by a variable name to make it constant:

#!/bin/bash
#Filename: constant.sh
#Description: constant variables in shell

readonly text="Welcome to Linux Shell Scripting"
echo $text
declare -r number=27
echo $number
text="Welcome"

The result after running this script will look as follows:

Welcome to Linux Shell Scripting
27
constant.sh: line 9: text: readonly variable

From the error message, it's clear that we can't change the value of a constant variable, and also we can't unset the value of the constant variable.

We can ask the user to provide input using the read shell built in command. The number of inputs to be given by a user is equivalent to the number of arguments provided to read. The value inserted by a user is stored in respective parameters passed to read. All parameters act as variables in which the corresponding user input value is stored.

The syntax of read is as follows:

read [options] var1 var2  … varN

If no variable in an argument is specified, the input value by a user will be stored in the inbuilt variable REPLY and can be accessed further using $REPLY.

We can read a user input in its input variable as follows:

$ read
    Hello World
$ echo $REPLY
    Hello World

We can read a value from user input as follows:

$ read text
    Hello
$ echo $text
    Hello

We can read multiple values from user input as follows:

$ read name usn marks
    Foo 345 78
$ echo $name $usn $marks
    Foo 345 78

We can read only the n characters and don't wait for the user to input a complete line as follows:

$ read -n 5    # option -n number takes only 5 characters from user input
    Hello$
$ echo $REPLY
    Hello

We can prompt the user a message before reading user input as follows:

$ read -p "What is your name?"    # -p allows to prompt user a message
    What is your name?Foo
$  echo $REPLY
    Foo

Hiding an input character when reading in console:

$  read -s -p "Enter your secret key:"  # -s doesn't echo input in console
Enter your secret key:$    #Pressing enter key brings command prompt $
echo $REPLY
foo

The following example shows the read command's usage:

#!/bin/bash
#Filename: read.sh
#Description: Find a file in a path entered by user

read -p "Enter filename to be searched:"
filename=$REPLY
read -p "Enter path for search:" path
echo "File $filename search matches to"
find $path -name $filename

The following is the result of running the read.sh script in bash:

Enter filename to be searched:read
Enter path for search:/usr/bin
File read search matches to
/usr/bin/read

Here, the find command has been used to search for the filename in the specified path. The detailed discussion of the command find will be done in Chapter 6, Working with Files.

Equal to an operator (=) is the assignment operator that is used to initialize or change the value of a variable. This operator works on any data such as a string, integer, float, array, and so on. For example:

$ var=40           # Initializing variable var to integer value
$ var="Hello"    # Changing value of var to string value
$ var=8.9        # Changing value of var to float value

Arithmetic operators are used for doing arithmetic operations on integers. They are as follows:

To perform any arithmetic operation, we prefix the expr and let keywords before the actual arithmetic expression. The following example shows how to perform an arithmetic operation in bash:

#!/bin/bash
#Filename: arithmetic.sh
#Description: Arithmetic evaluation

num1=10 num2=5
echo "Numbers are num1 = $num1 and num2 = $num2"
echo "Addition = `expr $num1 + $num2`"`"
echo "Subtraction = `expr $num1 - $num2`"
echo "Multiplication = `expr $num1 \* $num2`"
echo "Division = `expr $num1 / $num2`"
let "exponent = $num1 ** num2"
echo "Exponentiation = $exponent" 
echo "Modulo = `expr $num1 % $num2`"
let "num1 += $num2"
echo "New num1 = $num1"
let "num1 -= $num1"
echo "New num2 = $num2"

The result after running this script will look as follows:

Numbers are num1 = 10 and num2 = 5
Addition = 15
Subtraction = 5
Multiplication = 50
Division = 2
Exponentiation = 100000
Modulo = 0
New num1 = 15
New num2 = 5

Logical operators are also known as Boolean operators. They are:

! (NOT), && (AND), and || (OR)

Performing a logical operation returns a Boolean value as true (1) or false (0) depending upon the values of variable(s) on which the operation is done.

One of the useful use-case is: suppose that we want to execute a command if the first command or operation returns successfully. In this case, we can use the && operator. Similarly, if we want to execute another command, irrespective of the first command that got executed or not, then we can use the || operator between two commands. We can use the ! operator to negate the true value. For example:

$ cd ~/Documents/ && ls

The cd command is used to change the current path to the specified argument. Here, the cd ~/Documents/ command will change the directory to Documents if exists. If it fails, then ls won't get executed, but if cd to Documents succeeds, the ls command will display the content of Documents directory:

$ cat ~/file.txt  || echo "Current Working directory $PWD"
cat: /home/skumari/file.txt: No such file or directory
Current Working directory /tmp/

The cat command displays the content of file.txt if it exists. Irrespective of the cat ~/file.txt command execution, later the command that is echo "Current Working directory $PWD" will be executed:

$  ! cd /tmp/foo && mkdir /tmp/foo
bash: cd: /tmp/foo: No such file or directory

By running the preceding commands, first it will try to change the directory to /tmp/foo. Here, ! cd /tmp/foo means if change directory to /tmp/foo doesn't succeed, then run the second command, which is mkdir /tmp/foo. The mkdir command is used to create a new directory. As a result of proceeding command execution, directory /tmp/foo will be created if it doesn't exist.

$ cd /tmp/foo

Since the /tmp/foo directory has been created, a successful change of the directory will occur.

Comparison operators compare two variables and check whether a condition is satisfied or not. They are different for integers and strings.

Comparison operators that are valid for integer variables (consider a and b as two integer variables; for example, a=20, b=35) are as follows:

Comparison operators that are valid for string variables (consider a and b as two string variables; for example, a="Hello" b="World") are as follows:

Shell uses the < and > operators for redirection, so it should be used with an escape (\) if used under [ … ]. Double parentheses, (( ... )) or [[ … ]], doesn't need an escape sequence. Using [[ … ]] also supports pattern matching.

We will see the usage and examples of operators in more detail in Chapter 3, Effective Script Writing.

~ (Tilde) gets expanded by bash when it is present at the beginning of an unquoted string. The expansion depends upon what tilde-prefix is used. Tilde prefixes are characters until the first unquoted (/) slash. Some of the bash expansions are as follows:

It matches zero or more characters. Take a test directory as an example:

It matches any single character: ? (single question mark will match a single character), ?? (double question mark matches any two characters), and so on. Take a test directory as an example:

$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

This will recreate files that were removed during the previous example, and also update the access and modification time of the existing files:

We see only two files in the test directory because the rest of the files were of the length 3 or more.

Square brackets match any character from the characters mentioned inside the square brackets. Characters can be specified as a word or range.

A range of characters can be specified using - (hyphen). For example:

Take a test directory as an example and recreate files in a test directory:

$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

Get files whose name starts with a, b, c, or d with the following command:

$ ls [a-d]*
a  ab  bar  bar.c  bar.sh

Get files whose name starts with any letter and ends with a letter o or h, with the following command:

$  ls [a-zA-Z]*[oh]
foo  hello  hello.sh  moo  moo.sh

Get files that have at least the letter o twice in its name, with the following command:

$ ls *[o]*[o]*
foo  foo.c  foo.txt  moo  moo.c  moo.sh

[!characters] (Exclamation mark) is used to match a character that is not part of a charter set mentioned inside square brackets.

Get files that don't have a number in its name, with the following command:

$  ls [!0-9]*
a  ab  bar  bar.c  bar.sh  foo  foo.c  foo.txt  hello  hello.sh  hello.txt  moo  moo.c  moo.sh

It creates multiple wildcard patterns to match. A brace expression may contain either a comma-separated list of strings, a range, or a single character.

A range can be specified by using the following:

Take a test directory as an example and recreate files in the test directory:

$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

Get files that have the file extension .sh or .c, with the following command:

$ ls {*.sh,*.c}
bar.c  bar.sh  foo.c  hello.sh  moo.c  moo.sh

Copy bar.c to bar.html by using the following command:

$ cp bar{.c,.cpp}  # Expands to cp bar.c bar.cpp
$ ls bar.*
bar.c  bar.cpp  bar.sh

Print the number from 1 to 50 by using the following command:

$ echo {1..50}
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

Create 10 files that start with hello and has an extension .cpp:

$ touch hello{0..9}.cpp
$ ls *.cpp
hello0.cpp  hello1.cpp  hello2.cpp  hello3.cpp  hello4.cpp  hello5.cpp  hello6.cpp  hello7.cpp  hello8.cpp  hello9.cpp

To avoid shell expansion of a wildcard, use backslash (\) or write a string within a single quote (' ').

In a shell script, by default, the next line is executed if an error occurs in the current line. Sometimes, we may want to stop running a script further after an error has been encountered. The -e option of set ensures to exit a script once any of the commands in a pipeline fails.

In the following shell script, do_not_exit_on_failure.sh doesn't use set with the option -e:

$ cat do_not_exit_on_failure.sh

#!/bin/bash
# Filename: do_not_exit_on_failure.sh
# Description: Resume script after an error

echo "Before error"
cd /root/       # Will give error
echo "After error"

After running this script, the output is as follows:

Before error
do_not_exit_on_failure.sh: line 6: cd: /root/: Permission denied
After error

We see that the command after the error gets executed as well. In order to stop the execution after an error is encountered, use set -e in the script. The following script demonstrates the same:

$ cat exit_on_failure.sh

#!/bin/bash
# Filename: exit_on_failure.sh
# Description: Exits script after an error

set -e
echo "Before error"
cd /root/       # Will give error
echo "After error"

The output after running the preceding script is as follows:

Before error
exit_on_failure.sh: line 7: cd: /root/: Permission denied

We can see that the script has been terminated after encountering an error at the line number 7.

Using set with the -P option doesn't resolve symbolic links. Following example demonstrate how we can enable or disable symbolic link resolution of /bin directory which is symbolic link of /usr/bin/ directory:

$ ls -l /bin
lrwxrwxrwx. 1 root root 7 Nov 18 18:03 /bin -> usr/bin
$ set –P    # -P enable symbolic link resolution
$ cd /bin
$ pwd
/usr/bin
$ set +P   # Disable symbolic link resolution
$ pwd
/bin

We can use the set command to see all local variables accessible for the current process. The local variables are not accessible in the subprocess.

We can create our own variable and set it locally as follows:

$ MYVAR="Linux Shell Scripting"
$ echo $MYVAR
 Linux Shell Scripting
$ set | grep MYVAR  # MYVAR local variable is created
MYVAR='Linux Shell Scripting'
$ bash    # Creating a new bash sub-process in current bash
$ set | grep MYVAR
$    # Blank because MYVAR is local variable

To make a variable accessible to its subprocesses as well, use the export command followed by the variable to be exported:

$ MYVARIABLE="Hello World"
$ export  MYVARIABLE
$ bash    # Creating a new bash sub-process under bash
$ echo $MYVARIABLE
Hello World

This will export the MYVARIABLE variable to any subprocess that ran from that process. To check whether MYVARIABLE has exported or not, run the following command:

$ export |grep MYVARIABLE
declare -x MYVARIABLE="Hello World"
$ export | grep MYVAR
$MYVAR variable is not present in sub-process but variable MYVARIABLE is present in sub-process.

To unset local or exported variables, use the unset command and it will reset the value of the variable to null:

$ unset MYVAR        # Unsets local variable MYVAR
$ unset  MYVARIABLE    # Unsets exported variable MYVARIABLE