Book Image

Mastering Linux Shell Scripting

By : Andrew Mallett
Book Image

Mastering Linux Shell Scripting

By: Andrew Mallett

Overview of this book

Shell scripting is a quick method to prototype a complex application or a problem by automating tasks when working on Linux-based systems. Using both simple one-line commands and command sequences complex problems can be solved with ease, from text processing to backing up sysadmin tools. In this book, you’ll discover everything you need to know to master shell scripting and make informed choices about the elements you employ. Get to grips with the fundamentals of creating and running a script in normal mode, and in debug mode. Learn about various conditional statements' code snippets, and realize the power of repetition and loops in your shell script. Implement functions and edit files using the Stream Editor, script in Perl, program in Python – as well as complete coverage of other scripting languages to ensure you can choose the best tool for your project.
Table of Contents (21 chapters)
Mastering Linux Shell Scripting
About the Author
About the Reviewer

Chapter 11. Summarizing Logs with Awk

One of the tasks that awk is really good at is filtering data from log files. These log files may be many lines in length, perhaps 250,000 or more. I have worked with data with over a millions lines. Awk can process these lines quickly and effectively. As an example, we will work with a web server access log with 30,000 lines to show how effective and well written awk code can be. As we work our way through the chapter, we will also see different log files and review some of the techniques that we can employ with the awk command and the awk programming language to help with the reporting and administration of our services. In this chapter we will cover the following topics:

  • HTTPD log file format

  • Displaying data from web server logs

  • Summarizing HTTP access codes

  • Displaying the highest ranking client IP addresses

  • Listing browser data

  • Working with e-mail logs