One of the tasks that awk is really good at is filtering data from log files. These log files may be many lines in length, perhaps 250,000 or more. I have worked with data with over a millions lines. Awk can process these lines quickly and effectively. As an example, we will work with a web server access log with 30,000 lines to show how effective and well written awk code can be. As we work our way through the chapter, we will also see different log files and review some of the techniques that we can employ with the awk command and the awk programming language to help with the reporting and administration of our services. In this chapter we will cover the following topics:
HTTPD log file format
Displaying data from web server logs
Summarizing HTTP access codes
Displaying the highest ranking client IP addresses
Listing browser data
Working with e-mail logs