Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Linux Shell Scripting Cookbook, Second Edition
  • Table Of Contents Toc
Linux Shell Scripting Cookbook, Second Edition

Linux Shell Scripting Cookbook, Second Edition - Second Edition

4.3 (9)
Buy this Book
close
close
Linux Shell Scripting Cookbook, Second Edition

Linux Shell Scripting Cookbook, Second Edition

4.3 (9)
Buy this Book

Overview of this book

The shell remains one of the most powerful tools on a computer system — yet a large number of users are unaware of how much one can accomplish with it. Using a combination of simple commands, we will see how to solve complex problems in day to day computer usage.Linux Shell Scripting Cookbook, Second Edition will take you through useful real-world recipes designed to make your daily life easy when working with the shell. The book shows the reader how to effectively use the shell to accomplish complex tasks with ease.The book discusses basics of using the shell, general commands and proceeds to show the reader how to use them to perform complex tasks with ease.Starting with the basics of the shell, we will learn simple commands with their usages allowing us to perform operations on files of different kind. The book then proceeds to explain text processing, web interaction and concludes with backups, monitoring and other sysadmin tasks.Linux Shell Scripting Cookbook, Second Edition serves as an excellent guide to solving day to day problems using the shell and few powerful commands together to create solutions.
Table of Contents (16 chapters)
close
close
close
Linux Shell Scripting Cookbook
Icon
Linux Shell Scripting Cookbook
Credits
Icon
Credits
About the Authors
Icon
About the Authors
About the Reviewers
Icon
About the Reviewers
www.PacktPub.com
Icon
www.PacktPub.com
Preface
Icon
Preface
Lock Free Chapter
1
Shell Something Out
Icon
Shell Something Out
Icon
Introduction
Icon
Printing in the terminal
Icon
Playing with variables and environment variables
Icon
Function to prepend to environment variables
Icon
Math with the shell
Icon
Playing with file descriptors and redirection
Icon
Arrays and associative arrays
Icon
Visiting aliases
Icon
Grabbing information about the terminal
Icon
Getting and setting dates and delays
Icon
Debugging the script
Icon
Functions and arguments
Icon
Reading the output of a sequence of commands in a variable
Icon
Reading n characters without pressing the return key
Icon
Running a command until it succeeds
Icon
Field separators and iterators
Icon
Comparisons and tests
2
Have a Good Command
Icon
Have a Good Command
Icon
Introduction
Icon
Concatenating with cat
Icon
Recording and playing back of terminal sessions
Icon
Finding files and file listing
Icon
Playing with xargs
Icon
Translating with tr
Icon
Checksum and verification
Icon
Cryptographic tools and hashes
Icon
Sorting unique and duplicates
Icon
Temporary file naming and random numbers
Icon
Splitting files and data
Icon
Slicing filenames based on extension
Icon
Renaming and moving files in bulk
Icon
Spell checking and dictionary manipulation
Icon
Automating interactive input
Icon
Making commands quicker by running parallel processes
3
File In, File Out
Icon
File In, File Out
Icon
Introduction
Icon
Generating files of any size
Icon
The intersection and set difference (A-B) on text files
Icon
Finding and deleting duplicate files
Icon
Working with file permissions, ownership, and the sticky bit
Icon
Making files immutable
Icon
Generating blank files in bulk
Icon
Finding symbolic links and their targets
Icon
Enumerating file type statistics
Icon
Using loopback files
Icon
Creating ISO files and hybrid ISO
Icon
Finding the difference between files, patching
Icon
Using head and tail for printing the last or first 10 lines
Icon
Listing only directories – alternative methods
Icon
Fast command-line navigation using pushd and popd
Icon
Counting the number of lines, words, and characters in a file
Icon
Printing the directory tree
4
Texting and Driving
Icon
Texting and Driving
Icon
Introduction
Icon
Using regular expressions
Icon
Searching and mining a text inside a file with grep
Icon
Cutting a file column-wise with cut
Icon
Using sed to perform text replacement
Icon
Using awk for advanced text processing
Icon
Finding the frequency of words used in a given file
Icon
Compressing or decompressing JavaScript
Icon
Merging multiple files as columns
Icon
Printing the nth word or column in a file or line
Icon
Printing text between line numbers or patterns
Icon
Printing lines in the reverse order
Icon
Parsing e-mail addresses and URLs from text
Icon
Removing a sentence in a file containing a word
Icon
Replacing a pattern with text in all the files in a directory
Icon
Text slicing and parameter operations
5
Tangled Web? Not At All!
chevron up
Icon
Tangled Web? Not At All!
Icon
Introduction
Icon
Downloading from a web page
Icon
Downloading a web page as plain text
Icon
A primer on cURL
Icon
Accessing Gmail e-mails from the command line
Icon
Parsing data from a website
Icon
Image crawler and downloader
Icon
Web photo album generator
Icon
Twitter command-line client
Icon
Creating a "define" utility by using the Web backend
Icon
Finding broken links in a website
Icon
Tracking changes to a website
Icon
Posting to a web page and reading the response
6
The Backup Plan
Icon
The Backup Plan
Icon
Introduction
Icon
Archiving with tar
Icon
Archiving with cpio
Icon
Compressing data with gzip
Icon
Archiving and compressing with zip
Icon
Faster archiving with pbzip2
Icon
Creating filesystems with compression
Icon
Backup snapshots with rsync
Icon
Version control-based backup with Git
Icon
Creating entire disk images using fsarchiver
7
The Old-boy Network
Icon
The Old-boy Network
Icon
Introduction
Icon
Setting up the network
Icon
Let us ping!
Icon
Listing all the machines alive on a network
Icon
Running commands on a remote host with SSH
Icon
Transferring files through the network
Icon
Connecting to a wireless network
Icon
Password-less auto-login with SSH
Icon
Port forwarding using SSH
Icon
Mounting a remote drive at a local mount point
Icon
Network traffic and port analysis
Icon
Creating arbitrary sockets
Icon
Sharing an Internet connection
Icon
Basic firewall using iptables
8
Put on the Monitor's Cap
Icon
Put on the Monitor's Cap
Icon
Introduction
Icon
Monitoring disk usage
Icon
Calculating the execution time for a command
Icon
Collecting information about logged in users, boot logs, and boot failures
Icon
Listing the top 10 CPU consuming processes in an hour
Icon
Monitoring command outputs with watch
Icon
Logging access to files and directories
Icon
Logfile management with logrotate
Icon
Logging with syslog
Icon
Monitoring user logins to find intruders
Icon
Remote disk usage health monitor
Icon
Finding out active user hours on a system
Icon
Measuring and optimizing power usage
Icon
Monitoring disk activity
Icon
Checking disks and filesystems for errors
9
Administration Calls
Icon
Administration Calls
Icon
Introduction
Icon
Gathering information about processes
Icon
Killing processes and send or respond to signals
Icon
Sending messages to user terminals
Icon
Gathering system information
Icon
Using /proc for gathering information
Icon
Scheduling with cron
Icon
Writing and reading the MySQL database from Bash
Icon
User administration script
Icon
Bulk image resizing and format conversion
Icon
Taking screenshots from the terminal
Icon
Managing multiple terminals from one
1
Index
Icon
Index

Finding broken links in a website

Some people manually check every page on a website to search for broken links. It is feasible for websites having very few pages, but gets difficult when the number of pages become large. It becomes really easy if we can automate the process of finding broken links. We can find the broken links by using HTTP manipulation tools. Let's see how to do it.

Getting ready

To identify the links and find the broken ones from the links, we can use lynx and curl. It has an option, namely -traversal, which will recursively visit pages on the website and build a list of all hyperlinks in the website. We can use cURL to verify each of the links for whether they're broken or not.

How to do it...

Let's write a Bash script with the help of the curl command to find out the broken links on a web page:

#!/bin/bash 
#Filename: find_broken.sh
#Desc: Find broken links in a website

if [ $# -ne 1 ]; 
then 
  echo -e "$Usage: $0 URL\n" 
  exit 1; 
fi 


echo Broken links: 

mkdir /tmp...
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Linux Shell Scripting Cookbook, Second Edition
End of Section 5
Next Section
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful
Order Page Read Now

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon