10. Awk Fundamentals | Mastering Linux Shell Scripting

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents

Mastering Linux Shell Scripting

By : Andrew Mallett

4.3 (10)

Mastering Linux Shell Scripting

4.3 (10)

By: Andrew Mallett

Overview of this book

Shell scripting is a quick method to prototype a complex application or a problem by automating tasks when working on Linux-based systems. Using both simple one-line commands and command sequences complex problems can be solved with ease, from text processing to backing up sysadmin tools. In this book, you’ll discover everything you need to know to master shell scripting and make informed choices about the elements you employ. Get to grips with the fundamentals of creating and running a script in normal mode, and in debug mode. Learn about various conditional statements' code snippets, and realize the power of repetition and loops in your shell script. Implement functions and edit files using the Stream Editor, script in Perl, program in Python – as well as complete coverage of other scripting languages to ensure you can choose the best tool for your project.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Free Chapter

1. What and Why of Scripting with Bash

1. What and Why of Scripting with Bash

Bash vulnerabilities

The bash command hierarchy

Preparing text editors for scripting

Creating and executing scripts

Debugging your scripts

Summary

2. Creating Interactive Scripts

2. Creating Interactive Scripts

Using echo with options

Basic script using read

Script comments

Enhancing scripts with read prompts

Limiting the number of entered characters

Controlling the visibility of entered text

Enhancing learning with simple scripts

Summary

3. Conditions Attached

3. Conditions Attached

Simple decision paths using command-line lists

Verifying user input with lists

Using the test shell builtin

Creating conditional statements using if

Extending if with else

More conditions with elif

Using case statements

Script – building a front-end with grep

Summary

4. Creating Code Snippets

4. Creating Code Snippets

Abbreviations

Using code snippets

Summary

5. Alternative Syntax

5. Alternative Syntax

Recapping test

Providing parameter defaults

When in doubt – Quote!

Advanced test using [[

Arithmetic operations using ((

Summary

6. Iterating with Loops

6. Iterating with Loops

For loops

Controlling the loop

Reading input from files

Creating operator menus

Summary

7. Creating Building Blocks with Functions

7. Creating Building Blocks with Functions

Introducing functions

Passing parameters to functions

Returning values from functions

Using functions in menus

Summary

8. Introducing sed

8. Introducing sed

Using grep to display text

Using regular expressions

Understanding the basics of sed

Summary

9. Automating Apache Virtual Hosts

9. Automating Apache Virtual Hosts

Apache name-based Virtual Hosts

Automating Virtual Host creation

Summary

10. Awk Fundamentals

10. Awk Fundamentals

The history behind awk

Displaying and filtering content from files

Formatting output

Further filtering to display users by UID

Awk control files

Summary

11. Summarizing Logs with Awk

11. Summarizing Logs with Awk

The HTTPD log file format

Displaying data from web logs

Displaying the highest ranking IP address

Displaying the browser data

Working with e-mail logs

Summary

12. A Better lastlog with Awk

12. A Better lastlog with Awk

Using awk ranges to exclude data

Conditions based on the number of fields

Manipulating the awk record separator to report on XML data

Summary

13. Using Perl as a Bash Scripting Alternative

13. Using Perl as a Bash Scripting Alternative

What is Perl?

Hello World

Perl arrays

Conditional statements in Perl

Using functions within Perl

Summary

14. Using Python as a Bash Scripting Alternative

14. Using Python as a Bash Scripting Alternative

What is Python?

Saying Hello World the Python way

Pythonic arguments

Significant whitespace

Reading user input

Summary

Index

Index

Further filtering to display users by UID

We have been able to build our skills with awk piece by piece and what we have learned has been useful. We can take these tiny steps and add them to start creating something a little more usable. Perhaps, we want to print just the standard users; these are usually users higher than 500 or 1000 depending on your particular distribution.

On the Raspbian distribution that I am using for this book, standard users start with UID 1000. The UID is the third field. This is really a simple matter of using the value of the third field as the range operator. We can see this in the following example:

$ awk -F":" '$3 > 999 ' /etc/passwd

We can show users with UID 101 with the following command:

$ awk -F":" '$3 < 101 ' /etc/passwd

These just give you an idea of some of the possibilities available with awk. The reality is that we can play all day with our arithmetic comparison operators.

We have also seen that with some of these examples, the awk statements become...

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Mastering Linux Shell Scripting

Search

Your notes and bookmarks