Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

By : VINCENT BUMGARNER

Buy this Book

Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

By: VINCENT BUMGARNER

Buy this Book

Overview of this book

Splunk is a data collection, indexing, and visualization engine for operational intelligence. It's a powerful and versatile search and analysis engine that lets you investigate, troubleshoot, monitor, alert, and report on everything that's happening in your entire IT infrastructure from one location in real time. Splunk collects, indexes, and harnesses all the fast moving machine data generated by our applications, servers, and devices - physical, virtual, and in the cloud.Given a mountain of machine data, this book shows you exactly how to learn to use Splunk to make something useful from it. Depending on your needs, you can learn to search, transform, and display data, or learn to administer your Splunk installation, large or small. "Implementing Splunk: Big Data Reporting and Development for Operational Intelligence" will help you get your job done faster, whether you read from the beginning or jump to what you need to know today. New and experienced users alike will find nuggets of wisdom throughout.This book provides you with valuable examples and step-by-step instructions, showing you how to take advantage of everything Splunk has to offer you, to make the most out of your machine data."Implementing Splunk: Big Data Reporting and Development for Operational Intelligence" takes you on a journey right from inception to a fully functioning implementation of Splunk. Using a real-world data walkthrough, you'll be shown how to search effectively, create fields, build dashboards, reports, and package apps, manage your indexes, integrate into the enterprise, and extend Splunk. This practical implementation guide equips you with high-level knowledge for configuring, deploying, extending, and integrating Splunk. Depending on the goal and skills of the reader, enough topics are covered to get you on your way to dashboard guru, app developer, or enterprise administrator. This book uses examples curates reference, and sage advice to help you make the most of this incredibly powerful tool.

Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

Credits

About the Author

About the Reviewers

www.PacktPub.com

Preface

Free Chapter

The Splunk Interface

Using the time picker

Using the field picker

Using Manager

Summary

Understanding Search

Using search terms effectively

Boolean and grouping operators

Clicking to modify your search

Using fields to search

Using wildcards efficiently

All about time

Making searches faster

Sharing results with others

Saving searches for reuse

Creating alerts from searches

Summary

Tables, Charts, and Fields

About the pipe symbol

Using top to show common field values

Using stats to aggregate values

Using chart to turn data

Using timechart to show values over time

Working with fields

Summary

Simple XML Dashboards

The purpose of dashboards

Using wizards to build dashboards

Scheduling the generation of dashboards

Editing the XML directly

UI Examples app

Building forms

Summary

Advanced Search Examples

Using subsearches to find loosely related events

Using transaction

Determining concurrency

Calculating events per slice of time

Rebuilding top

Summary

Extending Search

Using tags to simplify search

Using event types to categorize results

Using lookups to enrich data

Using macros to reuse logic

Creating workflow actions

Using external commands

Summary

Working with Apps

Defining an app

Included apps

Installing apps

Building your first app

Editing navigation

Customizing the appearance of your app

Object permissions

App directory structure

Adding your app to Splunkbase

Summary

Building Advanced Dashboards

Reasons for working with advanced XML

Reasons for not working with advanced XML

Development process

Advanced XML structure

Converting simple XML to advanced XML

Module logic flow

Understanding layoutPanel

Reusing a query

Using intentions

Creating a custom drilldown

Third-party add-ons

Summary

Summary Indexes and CSV Files

Understanding summary indexes

When to use a summary index

When to not use a summary index

Populating summary indexes with saved searches

Using summary index events in a query

Using sistats, sitop, and sitimechart

How latency affects summary queries

How and when to backfill summary data

Reducing summary index size

Calculating top for a large time frame

Storing raw events in a summary index

Using CSV files to store transient data

Summary

Configuring Splunk

Locating Splunk configuration files

The structure of a Splunk configuration file

Configuration merging logic

An overview of Splunk .conf files

User interface resources

Summary

Advanced Deployments

Planning your installation

Splunk instance types

Common data sources

Sizing indexers

Planning redundancy

Working with multiple indexes

Deploying the Splunk binary

Using apps to organize configuration

Configuration distribution

Using LDAP for authentication

Using Single Sign On

Load balancers and Splunk

Multiple search heads

Summary

Extending Splunk

Writing a scripted input to gather data

Using Splunk from the command line

Querying Splunk via REST

Writing commands

Writing a scripted lookup to enrich data

Writing an event renderer

Writing a scripted alert action to process results

Summary

Index

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Using macros to reuse logic

A macro serves the purpose of replacing bits of search language with expanded phrases. Using macros can help you reuse logic and greatly reduce the length of queries.

Let's use one of our examples from Chapter 5, Advanced Search Examples, as our example case:

sourcetype="impl_splunk_web" user=mary 
  | transaction maxpause=5m user
  | stats avg(duration) avg(eventcount)

Creating a simple macro

Let's take the last two lines of our query and convert them to a macro. First, navigate to Manager | Advanced search | Advanced search | Search macros and click on New.

Walking through our fields, we have:

Destination app: This is where the macro will live.
Name: This is the name we will use in our searches.
Definition: This is the text that will be placed in our search.
Use eval-based definition?: If checked, the Definition string is treated as an eval statement instead of raw text. We'll use this option later.
The remaining fields are used if arguments are specified. We will...

Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

By : VINCENT BUMGARNER

Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

By: VINCENT BUMGARNER

Overview of this book

Related Content you might be interested in

Current Title:

Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

Using macros to reuse logic

Creating a simple macro