Bash Cookbook

Bash Cookbook

By : Ron Brash, Ganesh Sanjiv Naik

Buy this Book

Bash Cookbook

By: Ron Brash, Ganesh Sanjiv Naik

Buy this Book

Overview of this book

In Linux, one of the most commonly used and most powerful tools is the Bash shell. With its collection of engaging recipes, Bash Cookbook takes you through a series of exercises designed to teach you how to effectively use the Bash shell in order to create and execute your own scripts. The book starts by introducing you to the basics of using the Bash shell, also teaching you the fundamentals of generating any input from a command. With the help of a number of exercises, you will get to grips with the automation of daily tasks for sysadmins and power users. Once you have a hands-on understanding of the subject, you will move on to exploring more advanced projects that can solve real-world problems comprehensively on a Linux system. In addition to this, you will discover projects such as creating an application with a menu, beginning scripts on startup, parsing and displaying human-readable information, and executing remote commands with authentication using self-generated Secure Shell (SSH) keys. By the end of this book, you will have gained significant experience of solving real-world problems, from automating routine tasks to managing your systems and creating your own scripts.

Title Page

Packt Upsell

Contributors

Preface

Free Chapter

Crash Course in Bash

Getting started with Bash and CLI fundamentals

Creating and using basic variables

Hidden Bash variables and reserved words

Conditional logic using if, else, and elseif

Case/switch statements and loop constructs

Using functions and parameters

Including source files

Retrieving return codes and output

Linking commands, pipes, and input/output

Getting program input parameters

Getting additional information about commands

Summary

Acting Like a Typewriter and File Explorer

Introduction

Basic searching for strings and files

Using wildcards and regexes

Math and calculations in script

Striping/altering/sorting/deleting/searching strings with Bash only

Using SED and AWK to remove/replace substrings

Formatting your data/output using echo and printf

Readying your script for different languages with internationalization

Calculating statistics and reducing duplicates based on file contents

Using file attributes with conditional logic

Reading delimited data and altered output format

Understanding and Gaining File System Mastery

Introduction

Viewing files from various angles – head, tail, less, and more

Searching for files by name and/or extension

Creating a diff of two files and patching

Creating symbolic links and using them effectively

Crawling filesystem directories and printing a tree

Finding and deleting duplicate files or directories

Joining and splitting files at arbitrary positions

Generating datasets and random files of various size

Making a Script Behave Like a Daemon

Introduction

Running a program continuously (forever) using looping constructs or recursion

Keeping programs/scripts running after logoff

Invoking commands when they require permissions

Sanitizing user input and for repeatable results

Making a simple multi-level user menu using select

Generating and trapping signals for cleanup

Using temporary files and lock files in your program

Leveraging timeout when waiting for command completion

Creating a file-in-file-out program and running processes in parallel

Executing your script on startup

Scripts for System Administration Tasks

Introduction

Gathering and aggregating system information

Gathering network information and connectivity diagnostics

Configuring basic network connectivity

Monitoring directories and files

Compressing and archiving files

Rotating files from RAM to storage for log rotation

Using Linux iptables for a firewall

Accessing SQL databases remotely or locally

Creating SSH keys for password less remote access

Creating and configuring cron Jobs for task scheduling

Creating users and groups systematically

Scripts for Power Users

Introduction

Creating Syslog entries and generating an alarm

Backing up and erasing media, disks, and partitions with DD

Creating graphics and presentations on the CLI

Checking for file integrity and tampering

Mounting network file systems and retrieving files

Browsing the web from the CLI

Capturing network traffic headlessly

Finding binary dependencies

Fetching time from different locations

Encrypting/decrypting files from a script

Writing Bash to Win and Profit

Introduction

Creating a lame utility HTTP server

Parsing RSS feeds and output HTML

Scraping the web and collecting files

Making a simple IRC chat bot logger

Blocking IP addresses from failed SSH attempts

Playing and managing audio from Bash

Creating a simple NAT and DMZ firewall

Parsing a GitHub project and generate a report

Creating a poor man's incremental remote backup

Using Bash scripts to monitor udev input

Using Bash to monitor battery life and optimize it

Using chroot and restricted Bash shells to secure scripts

Advanced Scripting Techniques

Introduction

Calculating and reducing the runtime of a script

Writing one-line conditional statements and loops

Avoiding command not found warnings/errors and improving portability

Creating a config file and using it in tandem with your scripts

Improving your shell – GCC and command line colors

Adding aliases, and altering user paths/variables

Echoing output to raw terminal devices

Creating simple frontend GUIs for Bash scripts

Compiling and installing your own Bash shell

Recording terminal sessions for automation

Writing high-quality scripts by example

Other Books You May Enjoy

Leave a review - let other readers know what you think

Index

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Finding and deleting duplicate files or directories

At one point, we had already talked about checking to see if strings inside of a file were unique and if we could sort them, but we haven't yet performed a similar operation on files. However, before diving in, let's make some assumptions about what constitutes a duplicate file for the purpose of this recipe: a duplicate file is one that may have a different name, but the same contents as another.

One way to investigate the contents of a file would be to remove all white space and purely check the strings contained within, or we could merely use tools such as SHA512sum and MD5sum to generate a unique hash (think unique string full of gibberish) of the contents of the files. The general flow would be as follows:

Using this hash, we can compare the hash against a list of hashes already computed.
If the has matches, we have seen the contents of this file before and so we can delete it.
If the hash is new, we can record the entry and move onto...

Bash Cookbook

By : Ron Brash, Ganesh Sanjiv Naik

Bash Cookbook

By: Ron Brash, Ganesh Sanjiv Naik

Overview of this book

Related Content you might be interested in

Current Title:

Bash Cookbook

Learning Linux Shell Scripting

Linux Shell Scripting Cookbook

Linux Shell Scripting Bootcamp

Finding and deleting duplicate files or directories