Book Image

Linux Shell Scripting Cookbook, Second Edition - Second Edition

Book Image

Linux Shell Scripting Cookbook, Second Edition - Second Edition

Overview of this book

The shell remains one of the most powerful tools on a computer system — yet a large number of users are unaware of how much one can accomplish with it. Using a combination of simple commands, we will see how to solve complex problems in day to day computer usage.Linux Shell Scripting Cookbook, Second Edition will take you through useful real-world recipes designed to make your daily life easy when working with the shell. The book shows the reader how to effectively use the shell to accomplish complex tasks with ease.The book discusses basics of using the shell, general commands and proceeds to show the reader how to use them to perform complex tasks with ease.Starting with the basics of the shell, we will learn simple commands with their usages allowing us to perform operations on files of different kind. The book then proceeds to explain text processing, web interaction and concludes with backups, monitoring and other sysadmin tasks.Linux Shell Scripting Cookbook, Second Edition serves as an excellent guide to solving day to day problems using the shell and few powerful commands together to create solutions.

Related Content you might be interested in

Current Title:

Small book image
Linux Shell Scripting Cookbook, Second Edition - Second Edition
May, 2013 | 384 pages
No titles found
Table of Contents (16 chapters)
Linux Shell Scripting Cookbook
Linux Shell Scripting Cookbook
Credits
Credits
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Shell Something Out
Shell Something Out
Introduction
Printing in the terminal
Playing with variables and environment variables
Function to prepend to environment variables
Math with the shell
Playing with file descriptors and redirection
Arrays and associative arrays
Visiting aliases
Grabbing information about the terminal
Getting and setting dates and delays
Debugging the script
Functions and arguments
Reading the output of a sequence of commands in a variable
Reading n characters without pressing the return key
Running a command until it succeeds
Field separators and iterators
Comparisons and tests
2
Have a Good Command
Have a Good Command
Introduction
Concatenating with cat
Recording and playing back of terminal sessions
Finding files and file listing
Playing with xargs
Translating with tr
Checksum and verification
Cryptographic tools and hashes
Sorting unique and duplicates
Temporary file naming and random numbers
Splitting files and data
Slicing filenames based on extension
Renaming and moving files in bulk
Spell checking and dictionary manipulation
Automating interactive input
Making commands quicker by running parallel processes
3
File In, File Out
File In, File Out
Introduction
Generating files of any size
The intersection and set difference (A-B) on text files
Finding and deleting duplicate files
Working with file permissions, ownership, and the sticky bit
Making files immutable
Generating blank files in bulk
Finding symbolic links and their targets
Enumerating file type statistics
Using loopback files
Creating ISO files and hybrid ISO
Finding the difference between files, patching
Using head and tail for printing the last or first 10 lines
Listing only directories – alternative methods
Fast command-line navigation using pushd and popd
Counting the number of lines, words, and characters in a file
Printing the directory tree
4
Texting and Driving
Texting and Driving
Introduction
Using regular expressions
Searching and mining a text inside a file with grep
Cutting a file column-wise with cut
Using sed to perform text replacement
Using awk for advanced text processing
Finding the frequency of words used in a given file
Compressing or decompressing JavaScript
Merging multiple files as columns
Printing the nth word or column in a file or line
Printing text between line numbers or patterns
Printing lines in the reverse order
Parsing e-mail addresses and URLs from text
Removing a sentence in a file containing a word
Replacing a pattern with text in all the files in a directory
Text slicing and parameter operations
5
Tangled Web? Not At All!
Tangled Web? Not At All!
Introduction
Downloading from a web page
Downloading a web page as plain text
A primer on cURL
Accessing Gmail e-mails from the command line
Parsing data from a website
Image crawler and downloader
Web photo album generator
Twitter command-line client
Creating a "define" utility by using the Web backend
Finding broken links in a website
Tracking changes to a website
Posting to a web page and reading the response
6
The Backup Plan
The Backup Plan
Introduction
Archiving with tar
Archiving with cpio
Compressing data with gzip
Archiving and compressing with zip
Faster archiving with pbzip2
Creating filesystems with compression
Backup snapshots with rsync
Version control-based backup with Git
Creating entire disk images using fsarchiver
7
The Old-boy Network
The Old-boy Network
Introduction
Setting up the network
Let us ping!
Listing all the machines alive on a network
Running commands on a remote host with SSH
Transferring files through the network
Connecting to a wireless network
Password-less auto-login with SSH
Port forwarding using SSH
Mounting a remote drive at a local mount point
Network traffic and port analysis
Creating arbitrary sockets
Sharing an Internet connection
Basic firewall using iptables
8
Put on the Monitor's Cap
Put on the Monitor's Cap
Introduction
Monitoring disk usage
Calculating the execution time for a command
Collecting information about logged in users, boot logs, and boot failures
Listing the top 10 CPU consuming processes in an hour
Monitoring command outputs with watch
Logging access to files and directories
Logfile management with logrotate
Logging with syslog
Monitoring user logins to find intruders
Remote disk usage health monitor
Finding out active user hours on a system
Measuring and optimizing power usage
Monitoring disk activity
Checking disks and filesystems for errors
9
Administration Calls
Administration Calls
Introduction
Gathering information about processes
Killing processes and send or respond to signals
Sending messages to user terminals
Gathering system information
Using /proc for gathering information
Scheduling with cron
Writing and reading the MySQL database from Bash
User administration script
Bulk image resizing and format conversion
Taking screenshots from the terminal
Managing multiple terminals from one
Index
Index

The intersection and set difference (A-B) on text files

Intersection and set difference operations are commonly used in mathematical classes on set theory. However, similar operations on strings are also very helpful in some scenarios.

Getting ready

The comm command is a utility to perform a comparison between the two files. It has many good options to arrange the output in such a way that we can perform intersection, difference, and set difference operations.

  • Intersection: The intersection operation will print the lines that the specified files have in common with one another

  • Difference: The difference operation will print the lines that the specified files contain and that are not the same in all of those files

  • Set difference: The set difference operation will print the lines in file "A" that do not match those in all of the set of files specified ("B" plus "C" for example)

How to do it...

Note that comm takes only sorted files as input. Take a look at the following example:

$ cat A.txt
apple...
Previous Section
End of Section 4
Next Section