Book Image

Splunk Operational Intelligence Cookbook - Second Edition

By : Jose E. Hernandez, Josh Diakun, Derek Mock, Paul R. Johnson
Book Image

Splunk Operational Intelligence Cookbook - Second Edition

By: Jose E. Hernandez, Josh Diakun, Derek Mock, Paul R. Johnson

Overview of this book

Splunk makes it easy for you to take control of your data, and with Splunk Operational Cookbook, you can be confident that you are taking advantage of the Big Data revolution and driving your business with the cutting edge of operational intelligence and business analytics. With more than 70 recipes that demonstrate all of Splunk’s features, not only will you find quick solutions to common problems, but you’ll also learn a wide range of strategies and uncover new ideas that will make you rethink what operational intelligence means to you and your organization. You’ll discover recipes on data processing, searching and reporting, dashboards, and visualizations to make data shareable, communicable, and most importantly meaningful. You’ll also find step-by-step demonstrations that walk you through building an operational intelligence application containing vital features essential to understanding data and to help you successfully integrate a data-driven way of thinking in your organization. Throughout the book, you’ll dive deeper into Splunk, explore data models and pivots to extend your intelligence capabilities, and perform advanced searching to explore your data in even more sophisticated ways. Splunk is changing the business landscape, so make sure you’re taking advantage of it.
Table of Contents (17 chapters)
Splunk Operational Intelligence Cookbook Second Edition
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Preface
Index

Preface

Splunk makes it easy for you to take control of your data, and with Splunk Operational Cookbook, you can be confident that you are taking advantage of the Big Data revolution and driving your business with the cutting edge of operational intelligence and business analytics.

With more than 70 recipes that demonstrate all of Splunk's features, not only will you find quick solutions to common problems, but you'll also learn a wide range of strategies and uncover new ideas that will make you rethink what operational intelligence means to you and your organization.

What this book covers

Chapter 1, Play Time – Getting Data In, introduces you to the many ways in which you can get data into Splunk, whether it is collecting data locally from files and directories, receiving it through TCP/UDP port inputs, directly from a Universal Forwarder, or simply utilizing Scripted and Modular Inputs. Regardless of how Operational Intelligence is approached, the right data at the right time is pivotal to success; this chapter will play a key role in highlighting what data to consider and how to efficiently and effectively get that data into Splunk. It will also introduce the data sets that will be used throughout this book and where to obtain samples that can be used to follow each of the recipes as they are written.

Chapter 2, Diving into Data – Search and Report, introduces you to the first set of recipes in the book. Leveraging the data now available as a result of the previous chapter, the information and recipes will guide you through searching event data using Splunk's SPL (Search Processing Language); applying field extractions; grouping common events based on field values; and then building basic reports using the table, top, chart, and stats commands.

Chapter 3, Dashboards and Visualizations – Make Data Shine, guides you through building visualizations based on reports that can now be created as a result of this chapter. The information and recipes provided in this chapter will empower you to take their data, and reports, and bring it to life through the powerful visualizations provided by Splunk. Visualizations introduced will include single values, charts (bar, pie, line, and area), scatter charts, and gauges.

Chapter 4, Building an Operational Intelligence Application, builds on the understanding of visualizations that you gained as a result of the previous chapter to introduce the concept of dashboards. Dashboards provide a powerful way to bring visualizations together and provide the holistic visibility required to fully capture the operational intelligence that is most important. The information and recipes provided in this chapter will outline the purpose of dashboards, how to properly utilize dashboards, using the dashboard editor to build a dashboard, building a form for searching event data, and much more.

Chapter 5, Extending Intelligence – Data Models and Pivoting, covers one of the newest and most powerful features found in the latest release of Splunk Enterprise, the ability to create data models, and the introduction of the pivot tool. This chapter will take readers deeper into the data by introducing transactions, subsearching, concurrency, associations, and more advanced search commands. Through the information and recipes provided in this chapter readers will harness the ability to coverage data from different sources and understand or build relationships between the events.

Chapter 6, Diving Deeper – Advanced Searching, helps you harness the ability to coverage data from different sources and understand or build relationships between the events. By now you will have an understanding of how to derive operational intelligence from data by using some of Splunk's most common features. This chapter will introduce the concept of lookups and workflow actions for the purpose of augmenting the data being analyzed. The recipes provided will enable readers to apply this functionality to further enhance their understanding of the data being analyzed.

Chapter 7, Enriching Data – Lookups and Workflows, enables you to apply this functionality to further enhance their understanding of the data being analyzed. As illustrated in the preceding chapters, event data, whether from a single tier or multi-tier web application stack, can provide a wealth of operational intelligence and awareness. That intelligence can be further enriched through the use of lookups and workflow actions. This chapter will introduce readers to this concept, the benefits of proactive alerts and provide context of when alerts are best applied. The recipes provided will guide readers through creating alerts based on the knowledge gained from previous chapters.

Chapter 8, Being Proactive – Creating Alerts, guides you through creating alerts based on the knowledge gained from previous chapters. A key asset to complete operational intelligence and awareness is the ability to be proactive through scheduled or real-time alerts. This chapter will introduce readers to the concept of summary indexing for the purposes of accelerating reports and speeding up the time it takes to unlock business insight. The recipes in this chapter will provide readers with a short introduction to common situations where summary indexing can be leveraged to speed up reports or preserve focused statistics over long periods of time.

Chapter 9, Speed Up Intelligence – Data Summarization, provides you with a short introduction to common situations where summary indexing can be leveraged to speed up reports or preserve focused statistics over long periods of time. With big data being just that, big, it can sometimes be very time consuming searching massive sets of data and costly to store the data for long periods of time. This chapter will introduce readers to the concept of building data models and using the pivot tool to quickly design reports based on the data sets used within this book. With such powerful features now available, this chapter will enable readers to apply their extended knowledge of their data to empower non-traditional users to build intelligent operational reports through the use of data models and pivoting.

Chapter 10, Above and Beyond – Customization, Web Framework, REST API, HTTP Event Collector, and SDKs, introduces you to four very powerful features of Splunk, some of which are new to the latest release. These features provide the ability to create a very rich and powerful interactive experience with Splunk. This will open you up to the possibilities beyond core Splunk Enterprise and show you a method in order to create your own operational intelligence application that includes powerful D3 visualizations. It will also provide a recipe for querying Splunk's REST API and a basic Python application leveraging Splunk's SDK to execute a search.

What you need for this book

You'll need Splunk Enterprise 6.4 (or greater).

Who this book is for

This book is intended for users of all levels who are looking to leverage the Splunk Enterprise platform as a valuable operational intelligence tool. The recipes provided in this book will appeal to individuals from all facets of business, IT, security, product, marketing, and many more!

Also, existing users of Splunk who want to upgrade and get up and running with Splunk 6.4 will find this book invaluable.

Sections

In this book, you will find several headings that appear frequently (Getting ready, How to do it, How it works, There's more, and See also).

To give clear instructions on how to complete a recipe, we use these sections as follows:

Getting ready

This section tells you what to expect in the recipe, and describes how to set up any software or any preliminary settings required for the recipe.

How to do it…

This section contains the steps required to follow the recipe.

How it works…

This section usually consists of a detailed explanation of what happened in the previous section.

There's more…

This section consists of additional information about the recipe in order to make the reader more knowledgeable about the recipe.

See also

This section provides helpful links to other useful information for the recipe.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "Alternatively, if we want to search only for Windows logon events, we will search for eventtype=windows_logon."

A block of code is set as follows:

[WinEventLog://Application]
disabled = 0 
[WinEventLog://Security]
disabled = 0 
[WinEventLog://System]
disabled = 0 

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

index=main sourcetype="access_combined" | eval firsttime=_time | eval lasttime=_time | fields JSESSIONID firsttime lasttime | inputlookup session_state.csv append=true | stats last(firsttime) as firsttime, first(lasttime) as lasttime by JSESSIONID | outputlookup createinapp=true session_state.csv

Any command-line input or output is written as follows:

[monitor://c:\filelocation\cp01_messages.log]
sourcetype = linux_messages

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "Modular inputs are bundled as Splunk apps and, once installed, contain all the necessary configuration and code to display them in the Data inputs section of Splunk."

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail , and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

You can download the code files by following these steps:

  1. Log in or register to our website using your e-mail address and password.

  2. Hover the mouse pointer on the SUPPORT tab at the top.

  3. Click on Code Downloads & Errata.

  4. Enter the name of the book in the Search box.

  5. Select the book for which you're looking to download the code files.

  6. Choose from the drop-down menu where you purchased this book from.

  7. Click on Code Download.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

  • WinRAR / 7-Zip for Windows

  • Zipeg / iZip / UnRarX for Mac

  • 7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Splunk-Operational-Intelligence-Cookbook-Second-Edition. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Downloading the color images of this book

We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/SplunkOperationalIntelligenceCookbook_ColorImages.pdf.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at , and we will do our best to address the problem.