Book Image

Splunk Operational Intelligence Cookbook - Second Edition

By : Jose E. Hernandez, Josh Diakun, Derek Mock, Paul R. Johnson
Book Image

Splunk Operational Intelligence Cookbook - Second Edition

By: Jose E. Hernandez, Josh Diakun, Derek Mock, Paul R. Johnson

Overview of this book

Splunk makes it easy for you to take control of your data, and with Splunk Operational Cookbook, you can be confident that you are taking advantage of the Big Data revolution and driving your business with the cutting edge of operational intelligence and business analytics. With more than 70 recipes that demonstrate all of Splunk’s features, not only will you find quick solutions to common problems, but you’ll also learn a wide range of strategies and uncover new ideas that will make you rethink what operational intelligence means to you and your organization. You’ll discover recipes on data processing, searching and reporting, dashboards, and visualizations to make data shareable, communicable, and most importantly meaningful. You’ll also find step-by-step demonstrations that walk you through building an operational intelligence application containing vital features essential to understanding data and to help you successfully integrate a data-driven way of thinking in your organization. Throughout the book, you’ll dive deeper into Splunk, explore data models and pivots to extend your intelligence capabilities, and perform advanced searching to explore your data in even more sophisticated ways. Splunk is changing the business landscape, so make sure you’re taking advantage of it.

Related Content you might be interested in

Current Title:

Small book image
Splunk Operational Intelligence Cookbook - Second Edition
Jose E. Hernandez | Josh Diakun | Derek Mock | Paul R. Johnson
Jun, 2016 | 436 pages
No titles found
Table of Contents (17 chapters)
Splunk Operational Intelligence Cookbook Second Edition
Splunk Operational Intelligence Cookbook Second Edition
Credits
Credits
About the Authors
About the Authors
About the Reviewer
About the Reviewer
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Play Time – Getting Data In
Play Time – Getting Data In
Introduction
Indexing files and directories
Getting data through network ports
Using scripted inputs
Using modular inputs
Using the Universal Forwarder to gather data
Loading the sample data for this book
Defining field extractions
Defining event types and tags
2
Diving into Data – Search and Report
Diving into Data – Search and Report
Introduction
Making raw event data readable
Finding the most accessed web pages
Finding the most used web browsers
Identifying the top-referring websites
Charting web page response codes
Displaying web page response time statistics
Listing the top viewed products
Charting the application's functional performance
Charting the application's memory usage
Counting the total number of database connections
3
Dashboards and Visualizations – Making Data Shine
Dashboards and Visualizations – Making Data Shine
Introduction
Creating an Operational Intelligence dashboard
Using a pie chart to show the most accessed web pages
Displaying the unique number of visitors
Using a gauge to display the number of errors
Charting the number of method requests by type and host
Creating a timechart of method requests, views, and response times
Using a scatter chart to identify discrete requests by size and response time
Creating an area chart of the application's functional statistics
Using a bar chart to show the average amount spent by category
Creating a line chart of item views and purchases over time
4
Building an Operational Intelligence Application
Building an Operational Intelligence Application
Introduction
Creating an Operational Intelligence application
Adding dashboards and reports
Organizing the dashboards more efficiently
Dynamically drilling down on activity reports
Creating a form for searching web activity
Linking web page activity reports to the form
Displaying a geographical map of visitors
Scheduling PDF delivery of a dashboard
5
Extending Intelligence – Data Models and Pivoting
Extending Intelligence – Data Models and Pivoting
Introduction
Creating a data model for web access logs
Creating a data model for application logs
Accelerating data models
Pivoting total sales transactions
Pivoting purchases by geographic location
Pivoting slowest responding web pages
Pivot charting top error codes
6
Diving Deeper – Advanced Searching
Diving Deeper – Advanced Searching
Introduction
Calculating the average session time on a website
Calculating the average execution time for multi-tier web requests
Displaying the maximum concurrent checkouts
Analyzing the relationship of web requests
Predicting website traffic volumes
Finding abnormally-sized web requests
Identifying potential session spoofing
7
Enriching Data – Lookups and Workflows
Enriching Data – Lookups and Workflows
Introduction
Looking up product code descriptions
Flagging suspect IP addresses
Creating a session state table
Adding hostnames to IP addresses
Searching ARIN for a given IP address
Triggering a Google search for a given error
Creating a ticket for application errors
Looking up inventory from an external database
8
Being Proactive – Creating Alerts
Being Proactive – Creating Alerts
Introduction
Alerting on abnormal web page response times
Alerting on errors during checkout in real time
Alerting on abnormal user behavior
Alerting on failure and triggering a scripted response
Alerting when predicted sales exceed inventory
9
Speeding Up Intelligence – Data Summarization
Speeding Up Intelligence – Data Summarization
Introduction
Calculating an hourly count of sessions versus completed transactions
Backfilling the number of purchases by city
Displaying the maximum number of concurrent sessions over time
10
Above and Beyond – Customization, Web Framework, REST API, HTTP Event Collector, and SDKs
Above and Beyond – Customization, Web Framework, REST API, HTTP Event Collector, and SDKs
Introduction
Customizing the application navigation
Adding a force-directed graph of web hits
Adding a calendar heatmap of product purchases
Adding cell highlighting of average product price
Remotely querying Splunk's REST API for unique page views
Creating a Python application to return unique IP addresses
Creating a custom search command to format product names
Collecting data from remote scanning devices
Index
Index

About the Authors

Josh Diakun is an IT operations and security specialist with a focus on creating data-driven operational processes. He has over 10 years of experience managing and architecting enterprise-grade IT environments. For the past 7 years, he has been architecting, deploying and developing on Splunk as the core platform for organizations to gain security and operational intelligence. Josh is a founding partner at Discovered Intelligence, a company specializing in data intelligence services and solutions. He is also a co-founder of the Splunk Toronto User Group.

I would first like to thank my co-authors, Derek Mock and Paul Johnson, for their support, endless effort, and those many late nights that led to this book becoming a reality. To my partner, Rachel, an endless thank you for being my biggest supporter and making sure I always remembered to take a break. To my mother, Denyce, and sister, Jessika, thank you for being the two most amazing people in my life and cheering me on as I wrote this book. Finally to my late father, John, who was always an inspiration and brought the best out of me. Without this, I would not be where I am today.

Paul R Johnson has over 10 years of data intelligence experience in the areas of information security, operations, and compliance. He is a partner at Discovered Intelligence, a company specializing in data intelligence services and solutions. Paul previously worked for a Fortune 10 company, leading IT risk intelligence initiatives and managing a global Splunk deployment. Paul co-founded the Splunk Toronto User Group and lives and works in Toronto, Canada.

I would like to thank my fellow authors, Josh Diakun and Derek Mock, for their support and collaborative efforts in writing this book. Thanks guys for giving up nights, days, and weekends to get it completed! I would also like to thank my wife, Stacey, for her continuous support, for keeping me focused, and for her great feedback and patience.

Derek Mock is a software developer and big data architect who specializes in IT operations, information security, and cloud technologies. He has 15 years' experience developing and operating large enterprise-grade deployments and SaaS applications. He is a founding partner at Discovered Intelligence, a company specializing in data intelligence services and solutions. For the past 6 years, he has been leveraging Splunk as the core tool to deliver key operational intelligence. Derek is based in Toronto, Canada, and is a co-founder of the Splunk Toronto User Group.

I could not have asked for better co-authors than Josh Diakun and Paul Johnson, whose tireless efforts over many late nights brought this book into being. I would also like to thank my mentor, Dave Penny, for all his support in my professional life. Finally, thanks to my partner Alison, and my children, Sarah and James, for cheering me on as I wrote and for always making sure I had enough coffee.

Previous Section
Next Chapter