Splunk: Enterprise Operational Intelligence Delivered

By : Derek Mock, Betsy Page Sigman, Paul R. Johnson, Erickson Delgado, Josh Diakun, Ashish Kumar Tulsiram Yadav

Splunk: Enterprise Operational Intelligence Delivered

By: Derek Mock, Betsy Page Sigman, Paul R. Johnson, Erickson Delgado, Josh Diakun, Ashish Kumar Tulsiram Yadav

Overview of this book

Splunk is an extremely powerful tool for searching, exploring, and visualizing data of all types. Splunk is becoming increasingly popular, as more and more businesses, both large and small, discover its ease and usefulness. Analysts, managers, students, and others can quickly learn how to use the data from their systems, networks, web traffic, and social media to make attractive and informative reports. This course will teach everything right from installing and configuring Splunk. The first module is for anyone who wants to manage data with Splunk. You’ll start with very basics of Splunk— installing Splunk— before then moving on to searching machine data with Splunk. You will gather data from different sources, isolate them by indexes, classify them into source types, and tag them with the essential fields. With more than 70 recipes on hand in the second module that demonstrate all of Splunk’s features, not only will you find quick solutions to common problems, but you’ll also learn a wide range of strategies and uncover new ideas that will make you rethink what operational intelligence means to you and your organization. Dive deep into Splunk to find the most efficient solution to your data problems in the third module. Create the robust Splunk solutions you need to make informed decisions in big data machine analytics. From visualizations to enterprise integration, this well-organized high level guide has everything you need for Splunk mastery. This learning path combines some of the best that Packt has to offer into one complete, curated package. It includes content from the following Packt products: • Splunk Essentials - Second Edition • Splunk Operational Intelligence Cookbook - Second Edition • Advanced Splunk

Preface

What this learning path covers

What you need for this learning path

Who this learning path is for

Reader feedback

Customer support

Free Chapter

1. Module 1

1. Splunk in Action

2. Bringing in Data

3. Search Processing Language

4. Data Models and Pivot

5. Data Optimization, Reports, Alerts, and Accelerating Searches

6. Panes of Glass

7. Splunk SDK for JavaScript and D3.js

8. HTTP Event Collector

9. Best Practices and Advanced Queries

2. Module 2

1. Play Time – Getting Data In

2. Diving into Data – Search and Report

3. Dashboards and Visualizations – Making Data Shine

4. Building an Operational Intelligence Application

5. Extending Intelligence – Data Models and Pivoting

6. Diving Deeper – Advanced Searching

7. Enriching Data – Lookups and Workflows

8. Being Proactive – Creating Alerts

9. Speeding Up Intelligence – Data Summarization

10. Above and Beyond – Customization, Web Framework, REST API, HTTP Event Collector, and SDKs

3. Module 3

1. What's New in Splunk 6.3?

2. Developing an Application on Splunk

3. On-boarding Data in Splunk

4. Data Analytics

5. Advanced Data Analytics

6. Visualization

7. Advanced Visualization

8. Dashboard Customization

9. Advanced Dashboard Customization

10. Tweaking Splunk

11. Enterprise Integration with Splunk

12. What Next? Splunk 6.4

Biblography

Index

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Chapter 6. Panes of Glass

Splunk makes it easy to visualize data, and in this chapter we will show how to do that through a pane of glass or what is sometimes called a single pane of glass dashboard. This just means a dashboard that is designed with an effective graphic user interface that is easy to navigate and carefully laid out to be useful. This ability to make dashboards is one of Splunk's most useful features. Most of the charts that you will need to represent your data are already built into the application and you can organize these with very minimal effort. With practice, you can spin off a dashboard in a fraction of the time you might expect it to take.

In this chapter, we will work on the following topics:

How to identify the different types of dashboard
How to gather business requirements
How to modify dashboard panels
Building a single pane of glass dashboard with key performance indicators

Creating effective dashboards

It is easy to use Splunk to develop an attractive dashboard with multiple panels. However, this is not usually the best way to present your information. This is because a dashboard with multiple panels generally requires scrolling down the page, which can take the viewer's eyes away from crucial information. An effective dashboard should generally meet the following conditions:

Single page: The dashboard fits on a single page with no scrolling required
Multiple data points displayed: Charts and visualizations should display a number of data points
Crucial information highlighted: The dashboard points out the most important information, using appropriate titles, labels, legends, and markers
Created with the user in mind: Data is summarized in a way that is meaningful to the viewer
Loads quickly: The dashboard loads in a few seconds
Avoid redundancy: The display does not repeat information in multiple places

Types of dashboard

There are three kinds of dashboard that you will typically create with Splunk:

Dynamic form-based dashboards
Static real-time dashboards
Dashboards as scheduled reports

Dynamic form-based dashboards allow Splunk users to change the dashboard data without leaving the page. This is accomplished by adding input fields (such as time, radio (button), textbox, checkbox, dropdown, and so on) in the dashboard, which changes the data based on the current selection. This is an effective type of dashboard for teams that troubleshoot issues and analyze data.

Static real-time dashboards are often kept on a big panel screen for constant viewing, simply because they are so useful. You see these dashboards in data centers or Network Operations Centers (NOCs). Even though they are called static, in fact the data changes in real time without refreshing the page; it is just the format that stays constant. The dashboard will also have indicators and alerts that allow operators to easily identify a problem and act on it. Dashboards like this usually show the current state of the network or business systems, using indicators for web performance and traffic, revenue flow, and other important measures.

Dashboards as scheduled reports are the only kind that breaks away from the rules mentioned previously. This type will typically have multiple panels included on the same page. Also, the dashboard will not be exposed for viewing; it will generally be saved as a PDF file and sent to e-mail recipients at scheduled times. This format is ideal when you need to send information updates to multiple recipients at regular intervals.

In this chapter, we will create these three types of dashboard. You will also learn how to use and interact with the Splunk Dashboard Editor to develop advanced visualizations.

Gathering information and business requirements

Since you have signed in to the Splunk system with the user name Admin, you are considered a Splunk admin. With this exalted position come duties, one of which is being responsible for the data. As a custodian of data, a Splunk admin has a free rein over how to interpret and present the information to users. It is common that the administrator will be the one to create the first few dashboards. A more mature implementation, however, requires collaboration between different groups to create an output that is beneficial to all.

As a Splunk admin, make it a habit to consistently request feedback from your users regarding the dashboards. Without feedback, the dashboard that you created will only reflect your ideas and desires, not those of the rest of the team, whoever they may be. Sit down with the day-to-day users and lay out, on a drawing board, the business flow or the system diagrams to understand how things really work. Ask for users' stream-of-consciousness thoughts as you observe them use the dashboard in their work. Interview them and ask what data matters to them. Look for key phrases like these, which signify what data is most important to the business:

If this is broken, we lose tons of revenue...
This is an operations bottleneck and a constant point of failure...
We don't know what's going on here...
If only I can see the trend, it will make my work easier...
This is what my boss wants to see...

The people who will use your dashboard come from many areas of the business. You want to talk to all the different users, no matter where they are on the organizational chart. When you make friends with the architects, developers, business analysts, and management, you will end up building dashboards that benefit the entire organization.

We hope at this point that you have been convinced of the importance of dashboards and are ready to get started creating some, as we will do in the following sections.

Dynamic form-based dashboard

In this section, we will create a dynamic form-based dashboard in our Destinations app that will allow users to interact with form inputs to change and redisplay the data. Here is a screenshot of the final output of this dynamic form-based dashboard:

Dynamic dashboard with form input

Let's begin by creating the dashboard itself and then generating the base panels:

Open the Destinations app.
Run this search command:
```
SPL> index=main status_type="*" http_uri="*" server_ip="*" 
       | top status_type, status_description, http_uri, server_ip
```
Note
Important
Be careful when copying commands with quotation marks. It is best to type in the entire search command to avoid problems.
Click on Save As | Dashboard Panel.
Fill in the information based on the following screenshot:
Click on Save.
Close the pop-up window that appears.

Creating a Status Distribution panel

We will go to the dashboard later, once all our panel searches have been generated. Let us go ahead and create the second panel:

In the search window, type in the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
           | top status_type

You will save this as a dashboard panel to the newly-created dashboard. In the Dashboard option, click on the Existing button and look for the new dashboard, as seen here. Don't forget to fill in the Panel Title as Status Distribution:
Click on Save when you are done.

Creating the Status Types Over Time panel

Now we'll move on to create the next panel:

Type in the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
           | timechart count by http_status_code

You will save this as a Dynamic Form-based Dashboard panel as well. Type in Status Types Over Time in the Panel Title field:
Click on Save.

Creating the Hits vs Response Time panel

Now on to the next panel. Use the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
| timechart count, avg(http_response_time) as response_time

Save this dashboard panel as Hits vs Response Time:

Creating the Hits vs Response Time panel

Arranging the dashboard

Now, we'll go on to look at the dashboard we've created and make a few changes.

Click on the View Dashboard button. If you missed out on the View Dashboard button, you can find your dashboard by clicking on Dashboards in the main navigation bar.
Let us edit the panel arrangement. Click on Edit | Edit Panels.
Move the Status Distribution panel to the upper-right row.
Move the Hits vs Response Time panel to the lower-right row.
Click on Done to save your layout changes.

Look at the following screenshot. The dashboard framework you've created should now look much like this.

The dashboard probably looks a little plainer than you expected it to. But don't worry about how it looks for now. We will fix the dashboard one panel at a time.

Dynamic dashboard with four panels in tabular formats

Now that we have the layout framework in place, let us start modifying the panels. The first panel is how we want it to look so we do not need to change it.

Panel options

In this section, we will learn how to alter the look of our panels and create visualizations in them.

Go to Edit mode by clicking on Edit | Edit Panels.

Each dashboard panel will have three setting options to work with: inline search options, visualization type, and visualization options. They are represented by three dropdown icons.

The INLINE SEARCH drop-down allows you to modify the title, change the search string, change the time modifier for the search string, convert the panel into a report, and delete the panel.

The Visualization Type drop-down allows you to change the type of visualization to use for the panel, as shown in the following screenshot:

Finally, the Visualization Options drop-down will give you the ability to fine-tune your visualization. These options will change depending on the visualization you select. For a normal statistics table, this is how it will look.

Pie chart - status distribution

Go ahead and change the Status Distribution visualization panel to a pie chart. You do this by selecting the Visualization Type dropdown and selecting Pie. Once done, the panel will look like the following screenshot:

Stacked area chart - Status Types Over Time

Change the Status Types Over Time panel to Area. By default, area charts will not be stacked. Let us fix this by clicking on the Visualization Options dropdown.

In the Stack Mode section, click on Stacked. For Null Values, select Zero. Use the chart that follows for guidance.
Click on Apply. The panel will change right away.
Let us clean it up further. Let us remove the _time label as it is already implied. You can do this in the X-Axis section by setting the Title to None.

Here now is the new stacked area chart panel.

Stacked area chart - Status Types Over Time

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Gathering information and business requirements

If this is broken, we lose tons of revenue...
This is an operations bottleneck and a constant point of failure...
We don't know what's going on here...
If only I can see the trend, it will make my work easier...
This is what my boss wants to see...

We hope at this point that you have been convinced of the importance of dashboards and are ready to get started creating some, as we will do in the following sections.

Dynamic form-based dashboard

Dynamic dashboard with form input

Let's begin by creating the dashboard itself and then generating the base panels:

Open the Destinations app.
Run this search command:
```
SPL> index=main status_type="*" http_uri="*" server_ip="*" 
       | top status_type, status_description, http_uri, server_ip
```
Note
Important
Be careful when copying commands with quotation marks. It is best to type in the entire search command to avoid problems.
Click on Save As | Dashboard Panel.
Fill in the information based on the following screenshot:
Click on Save.
Close the pop-up window that appears.

Creating a Status Distribution panel

We will go to the dashboard later, once all our panel searches have been generated. Let us go ahead and create the second panel:

In the search window, type in the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
           | top status_type

You will save this as a dashboard panel to the newly-created dashboard. In the Dashboard option, click on the Existing button and look for the new dashboard, as seen here. Don't forget to fill in the Panel Title as Status Distribution:
Click on Save when you are done.

Creating the Status Types Over Time panel

Now we'll move on to create the next panel:

Type in the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
           | timechart count by http_status_code

You will save this as a Dynamic Form-based Dashboard panel as well. Type in Status Types Over Time in the Panel Title field:
Click on Save.

Creating the Hits vs Response Time panel

Now on to the next panel. Use the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
| timechart count, avg(http_response_time) as response_time

Save this dashboard panel as Hits vs Response Time:

Arranging the dashboard

Now, we'll go on to look at the dashboard we've created and make a few changes.

Click on the View Dashboard button. If you missed out on the View Dashboard button, you can find your dashboard by clicking on Dashboards in the main navigation bar.
Let us edit the panel arrangement. Click on Edit | Edit Panels.
Move the Status Distribution panel to the upper-right row.
Move the Hits vs Response Time panel to the lower-right row.
Click on Done to save your layout changes.

Look at the following screenshot. The dashboard framework you've created should now look much like this.

The dashboard probably looks a little plainer than you expected it to. But don't worry about how it looks for now. We will fix the dashboard one panel at a time.

Dynamic dashboard with four panels in tabular formats

Now that we have the layout framework in place, let us start modifying the panels. The first panel is how we want it to look so we do not need to change it.

Panel options

In this section, we will learn how to alter the look of our panels and create visualizations in them.

Go to Edit mode by clicking on Edit | Edit Panels.

Each dashboard panel will have three setting options to work with: inline search options, visualization type, and visualization options. They are represented by three dropdown icons.

The INLINE SEARCH drop-down allows you to modify the title, change the search string, change the time modifier for the search string, convert the panel into a report, and delete the panel.

The Visualization Type drop-down allows you to change the type of visualization to use for the panel, as shown in the following screenshot:

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Change the Status Types Over Time panel to Area. By default, area charts will not be stacked. Let us fix this by clicking on the Visualization Options dropdown.

In the Stack Mode section, click on Stacked. For Null Values, select Zero. Use the chart that follows for guidance.
Click on Apply. The panel will change right away.
Let us clean it up further. Let us remove the _time label as it is already implied. You can do this in the X-Axis section by setting the Title to None.

Here now is the new stacked area chart panel.

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Dynamic form-based dashboard

Dynamic dashboard with form input

Let's begin by creating the dashboard itself and then generating the base panels:

Open the Destinations app.
Run this search command:
```
SPL> index=main status_type="*" http_uri="*" server_ip="*" 
       | top status_type, status_description, http_uri, server_ip
```
Note
Important
Be careful when copying commands with quotation marks. It is best to type in the entire search command to avoid problems.
Click on Save As | Dashboard Panel.
Fill in the information based on the following screenshot:
Click on Save.
Close the pop-up window that appears.

Creating a Status Distribution panel

We will go to the dashboard later, once all our panel searches have been generated. Let us go ahead and create the second panel:

In the search window, type in the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
           | top status_type

You will save this as a dashboard panel to the newly-created dashboard. In the Dashboard option, click on the Existing button and look for the new dashboard, as seen here. Don't forget to fill in the Panel Title as Status Distribution:
Click on Save when you are done.

Creating the Status Types Over Time panel

Now we'll move on to create the next panel:

Type in the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
           | timechart count by http_status_code

You will save this as a Dynamic Form-based Dashboard panel as well. Type in Status Types Over Time in the Panel Title field:
Click on Save.

Creating the Hits vs Response Time panel

Now on to the next panel. Use the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
| timechart count, avg(http_response_time) as response_time

Save this dashboard panel as Hits vs Response Time:

Arranging the dashboard

Now, we'll go on to look at the dashboard we've created and make a few changes.

Click on the View Dashboard button. If you missed out on the View Dashboard button, you can find your dashboard by clicking on Dashboards in the main navigation bar.
Let us edit the panel arrangement. Click on Edit | Edit Panels.
Move the Status Distribution panel to the upper-right row.
Move the Hits vs Response Time panel to the lower-right row.
Click on Done to save your layout changes.

Look at the following screenshot. The dashboard framework you've created should now look much like this.

The dashboard probably looks a little plainer than you expected it to. But don't worry about how it looks for now. We will fix the dashboard one panel at a time.

Dynamic dashboard with four panels in tabular formats

Now that we have the layout framework in place, let us start modifying the panels. The first panel is how we want it to look so we do not need to change it.

Panel options

In this section, we will learn how to alter the look of our panels and create visualizations in them.

Go to Edit mode by clicking on Edit | Edit Panels.

Each dashboard panel will have three setting options to work with: inline search options, visualization type, and visualization options. They are represented by three dropdown icons.

The INLINE SEARCH drop-down allows you to modify the title, change the search string, change the time modifier for the search string, convert the panel into a report, and delete the panel.

The Visualization Type drop-down allows you to change the type of visualization to use for the panel, as shown in the following screenshot:

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Change the Status Types Over Time panel to Area. By default, area charts will not be stacked. Let us fix this by clicking on the Visualization Options dropdown.

In the Stack Mode section, click on Stacked. For Null Values, select Zero. Use the chart that follows for guidance.
Click on Apply. The panel will change right away.
Let us clean it up further. Let us remove the _time label as it is already implied. You can do this in the X-Axis section by setting the Title to None.

Here now is the new stacked area chart panel.

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Creating a Status Distribution panel

We will go to the dashboard later, once all our panel searches have been generated. Let us go ahead and create the second panel:

In the search window, type in the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
           | top status_type

You will save this as a dashboard panel to the newly-created dashboard. In the Dashboard option, click on the Existing button and look for the new dashboard, as seen here. Don't forget to fill in the Panel Title as Status Distribution:
Click on Save when you are done.

Creating the Status Types Over Time panel

Now we'll move on to create the next panel:

Type in the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
           | timechart count by http_status_code

You will save this as a Dynamic Form-based Dashboard panel as well. Type in Status Types Over Time in the Panel Title field:
Click on Save.

Creating the Hits vs Response Time panel

Now on to the next panel. Use the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
| timechart count, avg(http_response_time) as response_time

Save this dashboard panel as Hits vs Response Time:

Arranging the dashboard

Now, we'll go on to look at the dashboard we've created and make a few changes.

Click on the View Dashboard button. If you missed out on the View Dashboard button, you can find your dashboard by clicking on Dashboards in the main navigation bar.
Let us edit the panel arrangement. Click on Edit | Edit Panels.
Move the Status Distribution panel to the upper-right row.
Move the Hits vs Response Time panel to the lower-right row.
Click on Done to save your layout changes.

Look at the following screenshot. The dashboard framework you've created should now look much like this.

The dashboard probably looks a little plainer than you expected it to. But don't worry about how it looks for now. We will fix the dashboard one panel at a time.

Dynamic dashboard with four panels in tabular formats

Now that we have the layout framework in place, let us start modifying the panels. The first panel is how we want it to look so we do not need to change it.

Panel options

In this section, we will learn how to alter the look of our panels and create visualizations in them.

Go to Edit mode by clicking on Edit | Edit Panels.

Each dashboard panel will have three setting options to work with: inline search options, visualization type, and visualization options. They are represented by three dropdown icons.

The INLINE SEARCH drop-down allows you to modify the title, change the search string, change the time modifier for the search string, convert the panel into a report, and delete the panel.

The Visualization Type drop-down allows you to change the type of visualization to use for the panel, as shown in the following screenshot:

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Change the Status Types Over Time panel to Area. By default, area charts will not be stacked. Let us fix this by clicking on the Visualization Options dropdown.

In the Stack Mode section, click on Stacked. For Null Values, select Zero. Use the chart that follows for guidance.
Click on Apply. The panel will change right away.
Let us clean it up further. Let us remove the _time label as it is already implied. You can do this in the X-Axis section by setting the Title to None.

Here now is the new stacked area chart panel.

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Creating the Status Types Over Time panel

Now we'll move on to create the next panel:

Type in the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
           | timechart count by http_status_code

You will save this as a Dynamic Form-based Dashboard panel as well. Type in Status Types Over Time in the Panel Title field:
Click on Save.

Creating the Hits vs Response Time panel

Now on to the next panel. Use the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
| timechart count, avg(http_response_time) as response_time

Save this dashboard panel as Hits vs Response Time:

Arranging the dashboard

Now, we'll go on to look at the dashboard we've created and make a few changes.

Click on the View Dashboard button. If you missed out on the View Dashboard button, you can find your dashboard by clicking on Dashboards in the main navigation bar.
Let us edit the panel arrangement. Click on Edit | Edit Panels.
Move the Status Distribution panel to the upper-right row.
Move the Hits vs Response Time panel to the lower-right row.
Click on Done to save your layout changes.

Look at the following screenshot. The dashboard framework you've created should now look much like this.

The dashboard probably looks a little plainer than you expected it to. But don't worry about how it looks for now. We will fix the dashboard one panel at a time.

Dynamic dashboard with four panels in tabular formats

Now that we have the layout framework in place, let us start modifying the panels. The first panel is how we want it to look so we do not need to change it.

Panel options

In this section, we will learn how to alter the look of our panels and create visualizations in them.

Go to Edit mode by clicking on Edit | Edit Panels.

Each dashboard panel will have three setting options to work with: inline search options, visualization type, and visualization options. They are represented by three dropdown icons.

The INLINE SEARCH drop-down allows you to modify the title, change the search string, change the time modifier for the search string, convert the panel into a report, and delete the panel.

The Visualization Type drop-down allows you to change the type of visualization to use for the panel, as shown in the following screenshot:

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Change the Status Types Over Time panel to Area. By default, area charts will not be stacked. Let us fix this by clicking on the Visualization Options dropdown.

In the Stack Mode section, click on Stacked. For Null Values, select Zero. Use the chart that follows for guidance.
Click on Apply. The panel will change right away.
Let us clean it up further. Let us remove the _time label as it is already implied. You can do this in the X-Axis section by setting the Title to None.

Here now is the new stacked area chart panel.

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Creating the Hits vs Response Time panel

Now on to the next panel. Use the following search command:

SPL> index=main status_type="*" http_uri=* server_ip=* 
| timechart count, avg(http_response_time) as response_time

Save this dashboard panel as Hits vs Response Time:

Arranging the dashboard

Now, we'll go on to look at the dashboard we've created and make a few changes.

Click on the View Dashboard button. If you missed out on the View Dashboard button, you can find your dashboard by clicking on Dashboards in the main navigation bar.
Let us edit the panel arrangement. Click on Edit | Edit Panels.
Move the Status Distribution panel to the upper-right row.
Move the Hits vs Response Time panel to the lower-right row.
Click on Done to save your layout changes.

Look at the following screenshot. The dashboard framework you've created should now look much like this.

The dashboard probably looks a little plainer than you expected it to. But don't worry about how it looks for now. We will fix the dashboard one panel at a time.

Dynamic dashboard with four panels in tabular formats

Now that we have the layout framework in place, let us start modifying the panels. The first panel is how we want it to look so we do not need to change it.

Panel options

In this section, we will learn how to alter the look of our panels and create visualizations in them.

Go to Edit mode by clicking on Edit | Edit Panels.

Each dashboard panel will have three setting options to work with: inline search options, visualization type, and visualization options. They are represented by three dropdown icons.

The INLINE SEARCH drop-down allows you to modify the title, change the search string, change the time modifier for the search string, convert the panel into a report, and delete the panel.

The Visualization Type drop-down allows you to change the type of visualization to use for the panel, as shown in the following screenshot:

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Change the Status Types Over Time panel to Area. By default, area charts will not be stacked. Let us fix this by clicking on the Visualization Options dropdown.

In the Stack Mode section, click on Stacked. For Null Values, select Zero. Use the chart that follows for guidance.
Click on Apply. The panel will change right away.
Let us clean it up further. Let us remove the _time label as it is already implied. You can do this in the X-Axis section by setting the Title to None.

Here now is the new stacked area chart panel.

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Arranging the dashboard

Now, we'll go on to look at the dashboard we've created and make a few changes.

Click on the View Dashboard button. If you missed out on the View Dashboard button, you can find your dashboard by clicking on Dashboards in the main navigation bar.
Let us edit the panel arrangement. Click on Edit | Edit Panels.
Move the Status Distribution panel to the upper-right row.
Move the Hits vs Response Time panel to the lower-right row.
Click on Done to save your layout changes.

Look at the following screenshot. The dashboard framework you've created should now look much like this.

The dashboard probably looks a little plainer than you expected it to. But don't worry about how it looks for now. We will fix the dashboard one panel at a time.

Dynamic dashboard with four panels in tabular formats

Now that we have the layout framework in place, let us start modifying the panels. The first panel is how we want it to look so we do not need to change it.

Panel options

In this section, we will learn how to alter the look of our panels and create visualizations in them.

Go to Edit mode by clicking on Edit | Edit Panels.

Each dashboard panel will have three setting options to work with: inline search options, visualization type, and visualization options. They are represented by three dropdown icons.

The INLINE SEARCH drop-down allows you to modify the title, change the search string, change the time modifier for the search string, convert the panel into a report, and delete the panel.

The Visualization Type drop-down allows you to change the type of visualization to use for the panel, as shown in the following screenshot:

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Change the Status Types Over Time panel to Area. By default, area charts will not be stacked. Let us fix this by clicking on the Visualization Options dropdown.

In the Stack Mode section, click on Stacked. For Null Values, select Zero. Use the chart that follows for guidance.
Click on Apply. The panel will change right away.
Let us clean it up further. Let us remove the _time label as it is already implied. You can do this in the X-Axis section by setting the Title to None.

Here now is the new stacked area chart panel.

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Panel options

In this section, we will learn how to alter the look of our panels and create visualizations in them.

Go to Edit mode by clicking on Edit | Edit Panels.

Each dashboard panel will have three setting options to work with: inline search options, visualization type, and visualization options. They are represented by three dropdown icons.

The INLINE SEARCH drop-down allows you to modify the title, change the search string, change the time modifier for the search string, convert the panel into a report, and delete the panel.

The Visualization Type drop-down allows you to change the type of visualization to use for the panel, as shown in the following screenshot:

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Change the Status Types Over Time panel to Area. By default, area charts will not be stacked. Let us fix this by clicking on the Visualization Options dropdown.

In the Stack Mode section, click on Stacked. For Null Values, select Zero. Use the chart that follows for guidance.
Click on Apply. The panel will change right away.
Let us clean it up further. Let us remove the _time label as it is already implied. You can do this in the X-Axis section by setting the Title to None.

Here now is the new stacked area chart panel.

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Change the Status Types Over Time panel to Area. By default, area charts will not be stacked. Let us fix this by clicking on the Visualization Options dropdown.

In the Stack Mode section, click on Stacked. For Null Values, select Zero. Use the chart that follows for guidance.
Click on Apply. The panel will change right away.
Let us clean it up further. Let us remove the _time label as it is already implied. You can do this in the X-Axis section by setting the Title to None.

Here now is the new stacked area chart panel.

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Stacked area chart - Status Types Over Time

Change the Status Types Over Time panel to Area. By default, area charts will not be stacked. Let us fix this by clicking on the Visualization Options dropdown.

In the Stack Mode section, click on Stacked. For Null Values, select Zero. Use the chart that follows for guidance.
Click on Apply. The panel will change right away.
Let us clean it up further. Let us remove the _time label as it is already implied. You can do this in the X-Axis section by setting the Title to None.

Here now is the new stacked area chart panel.

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Column with line overlay combo chart - Hits vs Response Time

Now we are going to create a chart that combines a column and a line for the Hits vs. Response Time panel. This is good to use when representing two or more kinds of data with different ranges.

First change the chart panel visualization to Column.
In the Visualization Options dropdown, click on Chart Overlay.
In the Overlay selection box, select response_time.
Turn on View as Axis and click X-Axis.
Change Title to None.
Click on Legend.
Change the Legend Position to Bottom.
Click on Apply.
The new panel will now look similar to the following screenshot. Our data is not fluctuating much but imagine this with real data and you can easily spot trends and anomalies within the same time frame.
Click on Done to save all the changes you made and exit the Edit mode.
The dashboard has now come to life. This is how it should look now.
Dynamic form-based dashboard with four panels in different formats

Form inputs

Now that we have the dashboard layout that we want, it is time to make it dynamic and interactive. Before we proceed, let us just highlight some of the basic key concepts related to form inputs.

Just as in any web page, a form input is an element that allows you to select or type in information that will be submitted to the application for processing. There are different form inputs available for Splunk dashboards:

Text
Radio (which uses a radio button)
Dropdown (which uses a dropdown menu or list)
Checkbox
Multiselect (which allows you to select several responses)
Link list (this is a horizontal list that contains clickable links)
Time
Submit

If you click on Edit | Edit Panels, you will see that you can select which Form Input you require by clicking on the Add Input dropdown.

Go ahead and select Text in the Add Input drop-down. A new editable input field panel will be added at the very top of the dashboard. You can either edit the properties of the field using the pencil icon or delete the field entirely using the x icon.
Click on the pencil icon to edit the properties. You can change the type of input by clicking on the selections on the left of the pop-up window.

Although the options are generally different for every type of input, there are common concepts that you need to fully comprehend. So it is worth looking at this list carefully before we take you through some examples.

In the General section, you'll see the following options:

Label: Every input needs a label. This is what will be shown on the dashboard as the title of the input. An example of this would be: Select Sourcetype.
Search on Change: If checked, this checkbox triggers a reload of all panels that depend on the specific input. You typically want this checked, and want to be sure to include it if you think that the input will likely change.

In the Token Options section, you'll see the following option:

Token: This is an identifier for the field. It is used to associate the value that is returned by the form input against the identifier or token name. In programming, this is what you would refer to as a variable or ID. For example, if you created an input for time and you named the token time1, then in your panel's search query you can extract the value of the input field by calling the identifier $time1$ . Then the tokens that we use to get specific fields will be $time1$.earliest and $time1$.latest. You will walk through other examples in this chapter.
Default: On inputs that require a selection, you can specify a default value during page load. This is important if your panel charts require an initial value to populate the data. If not, your panels will not load data until the user selects an option.

In the Static Options section, you'll see the following option:

Name and Value: These are name-value pairs that will appear in the selection of the input field. For example, in a dropdown, if you have added a name called foo with the value bar, then foo will appear in the dropdown list but the value behind it is bar.

In the Dynamic Options section, you'll see the following options:

Search String: Occasionally the selection that you need shown in the input fields is already in Splunk. With this option, you can use a search query to populate the input field dynamically. For example, the search query index=main | top host will allow the input field to show all top hosts as a selectable option.
Time Range: This is the time range for the search query used previously. Try to use a small time range here.
Field for Label: This is the field that returns the value you need based on the search string. In the example previously, you need the field host.
Field for Value: You can change the field for the value but we recommend you use the same one as the label.

Creating a time range input

Let us change our input field into a time range field.

Click on Add Input.
On the list to the left, select Time.
In the General section, type Select Time Range in the Label space.
Click on the Search on Change checkbox.
Set the Default time range to Last 24 Hours.
Use the following screenshot as a guide.
Click Apply when done:
Before you save the dashboard changes, click the Autorun dashboard checkbox, as seen in the following screenshot:

You can now try to change the time range using the time input, but nothing will happen. This is because we have not yet configured the panels to react when the time input has been changed. Let us do that now:

Go back to Edit | Edit Panels mode.
Select Inline Search and edit Search String on the first panel.
Change Time Range Scope to Shared Time Picker (time).
Click on Save:

Notice that the data on the first panel now reacts to the changes you make on the time range input. Perform the same steps on the other three panels and watch the data change based on your selected time range.

Creating a radio input

Now we are going to create radio inputs with dynamic search options. This will allow viewers to select server and status types, and will affect the information rendered by the panels:

Click on Edit | Edit Panels.
Select Add Input | Radio.
Click on the Editicon in the newly created input.
In the Labelfield, type in Select Server.
Enable Search on Changeby checking the checkbox.
In the Token field, type server:
Scroll down to Static Optionsand click on it. In Static Options, add Name as ALLand Value as *.
Click Dynamic Options, then fill inSearch String, entering the following search command:
```
      SPL> index=main | top server_ip
```
Change the time range from All time to Last 60 minutes.
In Field For Label, type in server_ip.
In Field For Value, type in server_ip.
Now scroll back up to Token Options.
For Default, select ALL.
For Initial Value, select ALL.
Click Applyand you're done:

Now that you have configured the radio input with dynamic search options, you will see that the selection has been populated, along with the static option that you created. This is a great way of creating selection inputs when you know that the items will regularly change depending on a search query:

Try this exercise to reinforce what you have learned. Create a second radio input option, following the same steps as previously, with the following information:

Label: Select Status Type
Search on Change: Check
Token: status_type
Static Options: { Name: ALL, Value: * }
Search String: index=main | top status_type
Time Range: Last 60 minutes
Field For Label: status_type
Field For Value: status_type
Token Options Default: ALL
Token Options Initial Value: ALL

Click on Apply to save your changes.

If you did it correctly, the newly-created radio input will look like this:

Similar to when we first created the Time input, the panels will not react to these new inputs until we associate them. In order for the new inputs to change the values in the panels, we have to modify each panel search string to include the new tokens serverand status_type:

For the Status Types by URI panel, click on the Inline Search dropdown.
Select Edit Search String.

Carefully change the search string to match the following highlighted changes. This will filter the search results to show information for selected status_type and server_ip:

      SPL> index=main status_type="$status_type$" 
           http_uri=* server_ip=$server$ 
           | top status_type, status_description, http_uri, server_ip

Click on Save.
Then click on Apply to save the changes to the form input.
Click on Done to exit out of Edit mode.
Refresh the entire page using your browser's refresh icon.
Now change the selections of both the Select Server input and the Select Status Type input, and make sure the data on the first panel is changing.

Here is an example of data being filtered to 10.2.1.34 and Redirection for data arriving in the last 60 minutes:

At this point, you will appreciate what form input does to your dashboard. By simply substituting tokens in your search string, you are dynamically altering the panel charts so your users can filter the data in the ways they need. Continue editing the remaining panels using the following guide. You can refresh your browser if the changes do not happen right away.

Edit the Status Distribution panel to show the top ten status types:

       SPL> index=main status_type="$status_type$" 
            http_uri=* server_ip=$server$ 
            | top status_type

Edit the Status Over Time panel to show a timechart with counts reflecting status codes:

       SPL> index=main status_type="$status_type$" 
            http_uri=* server_ip=$server$ 
            | timechart count by http_status_code

Edit the Hits vs Response Time panel to show a timechart with counts for the number of events at each time and the average values for http_response_time for each time category (chosen by default, depending on the time span), labeled as response_time:
```
       SPL> index=main status_type="$status_type$" 
            http_uri=* server_ip=$server$ 
            | timechart count, avg(http_response_time) as response_time
```

Creating a dropdown input

Dropdown inputs function exactly the same as radio inputs. The former is used when the selection is huge and you do not want the list of choices to unnecessarily clutter the entire page. The http_uri field has numerous results, so this makes a drop-down the ideal candidate for input here.

Follow the same procedure as for radio input creation, but make sure you have selected Dropdown instead. Use the following information and screenshots as guides to complete the task:

Click on Edit | Edit Panels.
Select Add Input | Dropdown.
Click the Editicon in the newly created input.
In the Labelfield, type in Select HTTP URI to name your new drop-down.
As you did when you created a radio button, enable Search on Changeby checking the checkbox.
In the Token field, type http_uri.
Under Token Options section, in Default, select ALL.
Under Token Options section, in Initial Value, select ALL.
For Static Options, type { Name: ALL, Value: * }.
Under Dynamic Options, be sure the search icon is selected.
In the search string, type the following to designate that you want the index labeled main and top 0 to designate that you want to return all values of http_uri:
```
      SPL> index=main | top 0 http_uri
```
For the time range, specify Last 60 minutes.
In Field for Label, type http_uri.
In the Field for Value, also designatehttp_uri.
Click on Apply to save your changes:

If done correctly, the newly-created drop-down input will look like this:

Now that you have created the inputs, go ahead and associate them with the search panels. The same procedure applies; you have to edit each search string to include the new token:

Add the new dropdown token you have created to the Status Distribution panel, which will return the top 10 (by default) status types, along with their status descriptions, http_uri values, and server_ip values: http_uri=$http_uri$
```
      SPL> index=main status_type="$status_type$" http_uri=$http_uri$ 
           server_ip=$server$ | top status_type, status_description, 
           http_uri, server_ip
```

Also add the same token to the Status Over Time panel, which will then return a timechart of the top 10 counts for http_status_code:

      SPL> index=main status_type="$status_type$" http_uri=$http_uri$ 
           server_ip=$server$ | timechart count by http_status_code

And finally, add the token to the Hits vs Response Time panel, which will return a timechart showing the top 10 counts for average values of http_response_time (labeled as response_time):

      SPL> index=main status_type="$status_type$" http_uri=$http_uri$ 
           server_ip=$server$ | timechart count, avg(http_response_time) as 
           response_time

When all the form inputs are done, this is how it should look. We show the heading, where you can filter, first:

This is now a fully-functional, dynamic, form-based dashboard. It was a lot of work, but the more you do it, the easier the process becomes.

Dynamic form-based dashboard with four chart panels

Static Real-Time dashboard

In this section, we will create a real-time dashboard that will display crucial information based on the data we have. To encourage you, we present a screenshot here with how it will look when we are done:

Test real-time dashboard with advanced indicators, combo charts, and choropleth charts

Single Value Panels with color ranges

In the previous sections, you first created panels by running a search then saving them in a dashboard. After you created all your search panels, you then started to modify the visualization from the dashboard. This is one way to achieve it. However, typically you first want to see your visualization before adding it to a dashboard. This makes it a more straightforward approach. We will use that method in upcoming sections:

Let's start with a search command in the Destinations app to create the dashboard:

      SPL> index=main http_uri=/booking/confirmation http_status_code=200 
           | stats count

Select Real-Time > 1 hour window in the Time Range preset and run the command.
Click on the Visualizationtab to switch to visualization mode.
Click on the Format dropdown.
In the Under Label field, type last 60 mins.
In the Color tab, click on Yes to Use Colors.
Arrange the color ranges to match the following screenshot:
Click on Apply.
Click on Save As | Dashboard panel.
Select New dashboard and fill in the following information:
Click on Save.
Click on View Dashboard.

The panel you just created is set to Real Time search and will continuously update as long as the page is in view and you have not exceeded your real-time search quota.

Creating panels by cloning

There will be multiple occasions when you will need the same visualization for a different set of data. A very cool way of quickly doing this is by cloning previously created panels. We will create another color-coded single value panel by cloning the first one we created:

In your Real Time dashboard, go to edit mode by clicking on Edit | Edit Panels.
Click on Add Panel. The Add Panel slide-through will appear.
Expand Clone from Dashboard.
Expand Real Time Dashboard.
Click on Bookings. Use the following screenshot as a guide:
In the Preview pane, click on Add to Dashboard.
Click the title of the second Bookings panel and rename it Reservations.
Click on the Search modification dropdown and select Edit Search String.

Change the Search String to the following command:

      SPL> index=main http_uri=/booking/reservation http_status_code=200 
           | stats count

Click on Save.
Drag the second panel to the right of the first row.
Click on Done to save your settings.

You have successfully cloned a panel and shortened dashboard creation by a number of steps.

Single Value Panels with trends

We will now create two more single value panels that indicate trend lines. This is useful when you need your viewer to understand the behavior of the data in a very compressed line chart while highlighting the most current value. This is commonly used in viewing financial stock prices:

Enter edit mode with Edit | Edit Panels.
Create a clone of the Bookings panel. Follow the steps in the previous section.
Add it to the dashboard.
Rename the new panel Errors.

Change the Search String to the following command:

      SPL> index=main http_status_code=5* | timechart count

Click on the Visualization Options dropdown.
Set Under Label to last 60 mins.
Click on Yesfor Show Trend Indicator.
In the Compared to dropdown, select 1 hour before.
Ensure that Show Sparkline is set to Yes.
Refer to the following screenshot:
Click on Apply.
Click on the Search dropdown | Edit Search String.
In Select Time Range, click Real-time.
Change the Earliest value to 24 Hours Ago:
Click on Apply, then Save.
Drag the panel to the right end of the first row.
After following the previous steps, click on the Done button.

Repeat the previous procedure to create another panel. Use the following information to build the new panel:

Title: Response Time
Search String: index=main | timechart avg(http_response_time) as response_time span=1h
Time Range: Real-time 24 Hours Ago
After Label: ms
Under Label: compared to an hour ago
Show Trend in: Percent

The new single value panels have been created and are packed with information. First you see the current value within the last hour, then you see an upward or downward trend, and finally you see a sparkline (or trend line) that spans 24 hours.

The first row will now look similar to the following screenshot:

Real-time column charts with line overlays

It is time to build the second row of your real-time dashboard. Once again, we will use the cloning panel function:

Enter edit mode with Edit | Edit Panels.
Click Add Panel.
Clone the dynamic form-based dashboard: Hits vs Response Time:
Click Add to Dashboard.
Tip
Do not be surprised if the graph is not generated. Remember we are cloning from a form-based dashboard with input tokens.
Rename the panel to Traffic and Performance.

Change the Search String to remove input token references:

      SPL> index=main status_type="*" http_uri=* server_ip=* 
           | timechart count, avg(http_response_time) as response_time

Change the Time Range Scope to Explicit Selection.
Change the Time Range to Real-time and set its value as 24 Hours Ago.
Click on Save.
The chart will now populate the data.
Click on Done.

We will create another panel similar to the one we made previously. But this time, we will clone from the previous one to make our task easier.

Reload the browser. This is needed to load the newly-created panels in the clone selections.
Enter edit mode by Edit | Edit Panels.
Clone the Hits vs Response Time panel:
Rename the new panel to Booking Conversion.

Change the Search String:

     SPL> index=main http_uri=/booking/reservation OR 
          http_uri=/booking/confirmation 
          | timechart count by http_uri | rename /booking/confirmation AS 
          Confirmation, /booking/reservation AS Reservation 
          | eval Conversion=Reservation/Confirmation 
          | fields _time, Reservation, Confirmation, Conversion

Change Time Range to Real-time | 24 Hours Ago.
Click the Visualization Options dropdown.
Select the second option in Stack Mode as stacked.
Click Chart Overlay.
Delete the response_time overlay.
Add the Conversionoverlay.
Turn on View as Axis.
Click on Apply.
Drag this panel to the right of the second row.

You have completed the real-time version of the combo charts. It should look similar to the following screenshot:

Real-time column charts with line overlays

Real-time combo charts with line overlays>

Single Value Panels with color ranges

Let's start with a search command in the Destinations app to create the dashboard:

      SPL> index=main http_uri=/booking/confirmation http_status_code=200 
           | stats count

Select Real-Time > 1 hour window in the Time Range preset and run the command.
Click on the Visualizationtab to switch to visualization mode.
Click on the Format dropdown.
In the Under Label field, type last 60 mins.
In the Color tab, click on Yes to Use Colors.
Arrange the color ranges to match the following screenshot:
Click on Apply.
Click on Save As | Dashboard panel.
Select New dashboard and fill in the following information:
Click on Save.
Click on View Dashboard.

The panel you just created is set to Real Time search and will continuously update as long as the page is in view and you have not exceeded your real-time search quota.

Creating panels by cloning

In your Real Time dashboard, go to edit mode by clicking on Edit | Edit Panels.
Click on Add Panel. The Add Panel slide-through will appear.
Expand Clone from Dashboard.
Expand Real Time Dashboard.
Click on Bookings. Use the following screenshot as a guide:
In the Preview pane, click on Add to Dashboard.
Click the title of the second Bookings panel and rename it Reservations.
Click on the Search modification dropdown and select Edit Search String.

Change the Search String to the following command:

      SPL> index=main http_uri=/booking/reservation http_status_code=200 
           | stats count

Click on Save.
Drag the second panel to the right of the first row.
Click on Done to save your settings.

You have successfully cloned a panel and shortened dashboard creation by a number of steps.

Single Value Panels with trends

Enter edit mode with Edit | Edit Panels.
Create a clone of the Bookings panel. Follow the steps in the previous section.
Add it to the dashboard.
Rename the new panel Errors.

Change the Search String to the following command:

      SPL> index=main http_status_code=5* | timechart count

Click on the Visualization Options dropdown.
Set Under Label to last 60 mins.
Click on Yesfor Show Trend Indicator.
In the Compared to dropdown, select 1 hour before.
Ensure that Show Sparkline is set to Yes.
Refer to the following screenshot:
Click on Apply.
Click on the Search dropdown | Edit Search String.
In Select Time Range, click Real-time.
Change the Earliest value to 24 Hours Ago:
Click on Apply, then Save.
Drag the panel to the right end of the first row.
After following the previous steps, click on the Done button.

Repeat the previous procedure to create another panel. Use the following information to build the new panel:

Title: Response Time
Search String: index=main | timechart avg(http_response_time) as response_time span=1h
Time Range: Real-time 24 Hours Ago
After Label: ms
Under Label: compared to an hour ago
Show Trend in: Percent

The first row will now look similar to the following screenshot:

Real-time column charts with line overlays

It is time to build the second row of your real-time dashboard. Once again, we will use the cloning panel function:

Enter edit mode with Edit | Edit Panels.
Click Add Panel.
Clone the dynamic form-based dashboard: Hits vs Response Time:
Click Add to Dashboard.
Tip
Do not be surprised if the graph is not generated. Remember we are cloning from a form-based dashboard with input tokens.
Rename the panel to Traffic and Performance.

Change the Search String to remove input token references:

      SPL> index=main status_type="*" http_uri=* server_ip=* 
           | timechart count, avg(http_response_time) as response_time

Change the Time Range Scope to Explicit Selection.
Change the Time Range to Real-time and set its value as 24 Hours Ago.
Click on Save.
The chart will now populate the data.
Click on Done.

We will create another panel similar to the one we made previously. But this time, we will clone from the previous one to make our task easier.

Reload the browser. This is needed to load the newly-created panels in the clone selections.
Enter edit mode by Edit | Edit Panels.
Clone the Hits vs Response Time panel:
Rename the new panel to Booking Conversion.

Change the Search String:

     SPL> index=main http_uri=/booking/reservation OR 
          http_uri=/booking/confirmation 
          | timechart count by http_uri | rename /booking/confirmation AS 
          Confirmation, /booking/reservation AS Reservation 
          | eval Conversion=Reservation/Confirmation 
          | fields _time, Reservation, Confirmation, Conversion

Change Time Range to Real-time | 24 Hours Ago.
Click the Visualization Options dropdown.
Select the second option in Stack Mode as stacked.
Click Chart Overlay.
Delete the response_time overlay.
Add the Conversionoverlay.
Turn on View as Axis.
Click on Apply.
Drag this panel to the right of the second row.

You have completed the real-time version of the combo charts. It should look similar to the following screenshot:

Real-time combo charts with line overlays>

Creating panels by cloning

In your Real Time dashboard, go to edit mode by clicking on Edit | Edit Panels.
Click on Add Panel. The Add Panel slide-through will appear.
Expand Clone from Dashboard.
Expand Real Time Dashboard.
Click on Bookings. Use the following screenshot as a guide:
In the Preview pane, click on Add to Dashboard.
Click the title of the second Bookings panel and rename it Reservations.
Click on the Search modification dropdown and select Edit Search String.

Change the Search String to the following command:

      SPL> index=main http_uri=/booking/reservation http_status_code=200 
           | stats count

Click on Save.
Drag the second panel to the right of the first row.
Click on Done to save your settings.

You have successfully cloned a panel and shortened dashboard creation by a number of steps.

Single Value Panels with trends

Enter edit mode with Edit | Edit Panels.
Create a clone of the Bookings panel. Follow the steps in the previous section.
Add it to the dashboard.
Rename the new panel Errors.

Change the Search String to the following command:

      SPL> index=main http_status_code=5* | timechart count

Click on the Visualization Options dropdown.
Set Under Label to last 60 mins.
Click on Yesfor Show Trend Indicator.
In the Compared to dropdown, select 1 hour before.
Ensure that Show Sparkline is set to Yes.
Refer to the following screenshot:
Click on Apply.
Click on the Search dropdown | Edit Search String.
In Select Time Range, click Real-time.
Change the Earliest value to 24 Hours Ago:
Click on Apply, then Save.
Drag the panel to the right end of the first row.
After following the previous steps, click on the Done button.

Repeat the previous procedure to create another panel. Use the following information to build the new panel:

Title: Response Time
Search String: index=main | timechart avg(http_response_time) as response_time span=1h
Time Range: Real-time 24 Hours Ago
After Label: ms
Under Label: compared to an hour ago
Show Trend in: Percent

The first row will now look similar to the following screenshot:

Real-time column charts with line overlays

It is time to build the second row of your real-time dashboard. Once again, we will use the cloning panel function:

Enter edit mode with Edit | Edit Panels.
Click Add Panel.
Clone the dynamic form-based dashboard: Hits vs Response Time:
Click Add to Dashboard.
Tip
Do not be surprised if the graph is not generated. Remember we are cloning from a form-based dashboard with input tokens.
Rename the panel to Traffic and Performance.

Change the Search String to remove input token references:

      SPL> index=main status_type="*" http_uri=* server_ip=* 
           | timechart count, avg(http_response_time) as response_time

Change the Time Range Scope to Explicit Selection.
Change the Time Range to Real-time and set its value as 24 Hours Ago.
Click on Save.
The chart will now populate the data.
Click on Done.

We will create another panel similar to the one we made previously. But this time, we will clone from the previous one to make our task easier.

Reload the browser. This is needed to load the newly-created panels in the clone selections.
Enter edit mode by Edit | Edit Panels.
Clone the Hits vs Response Time panel:
Rename the new panel to Booking Conversion.

Change the Search String:

     SPL> index=main http_uri=/booking/reservation OR 
          http_uri=/booking/confirmation 
          | timechart count by http_uri | rename /booking/confirmation AS 
          Confirmation, /booking/reservation AS Reservation 
          | eval Conversion=Reservation/Confirmation 
          | fields _time, Reservation, Confirmation, Conversion

Change Time Range to Real-time | 24 Hours Ago.
Click the Visualization Options dropdown.
Select the second option in Stack Mode as stacked.
Click Chart Overlay.
Delete the response_time overlay.
Add the Conversionoverlay.
Turn on View as Axis.
Click on Apply.
Drag this panel to the right of the second row.

You have completed the real-time version of the combo charts. It should look similar to the following screenshot:

Real-time combo charts with line overlays>

Single Value Panels with trends

Enter edit mode with Edit | Edit Panels.
Create a clone of the Bookings panel. Follow the steps in the previous section.
Add it to the dashboard.
Rename the new panel Errors.

Change the Search String to the following command:

      SPL> index=main http_status_code=5* | timechart count

Click on the Visualization Options dropdown.
Set Under Label to last 60 mins.
Click on Yesfor Show Trend Indicator.
In the Compared to dropdown, select 1 hour before.
Ensure that Show Sparkline is set to Yes.
Refer to the following screenshot:
Click on Apply.
Click on the Search dropdown | Edit Search String.
In Select Time Range, click Real-time.
Change the Earliest value to 24 Hours Ago:
Click on Apply, then Save.
Drag the panel to the right end of the first row.
After following the previous steps, click on the Done button.

Repeat the previous procedure to create another panel. Use the following information to build the new panel:

Title: Response Time
Search String: index=main | timechart avg(http_response_time) as response_time span=1h
Time Range: Real-time 24 Hours Ago
After Label: ms
Under Label: compared to an hour ago
Show Trend in: Percent

The first row will now look similar to the following screenshot:

Real-time column charts with line overlays

It is time to build the second row of your real-time dashboard. Once again, we will use the cloning panel function:

Enter edit mode with Edit | Edit Panels.
Click Add Panel.
Clone the dynamic form-based dashboard: Hits vs Response Time:
Click Add to Dashboard.
Tip
Do not be surprised if the graph is not generated. Remember we are cloning from a form-based dashboard with input tokens.
Rename the panel to Traffic and Performance.

Change the Search String to remove input token references:

      SPL> index=main status_type="*" http_uri=* server_ip=* 
           | timechart count, avg(http_response_time) as response_time

Change the Time Range Scope to Explicit Selection.
Change the Time Range to Real-time and set its value as 24 Hours Ago.
Click on Save.
The chart will now populate the data.
Click on Done.

We will create another panel similar to the one we made previously. But this time, we will clone from the previous one to make our task easier.

Reload the browser. This is needed to load the newly-created panels in the clone selections.
Enter edit mode by Edit | Edit Panels.
Clone the Hits vs Response Time panel:
Rename the new panel to Booking Conversion.

Change the Search String:

     SPL> index=main http_uri=/booking/reservation OR 
          http_uri=/booking/confirmation 
          | timechart count by http_uri | rename /booking/confirmation AS 
          Confirmation, /booking/reservation AS Reservation 
          | eval Conversion=Reservation/Confirmation 
          | fields _time, Reservation, Confirmation, Conversion

Change Time Range to Real-time | 24 Hours Ago.
Click the Visualization Options dropdown.
Select the second option in Stack Mode as stacked.
Click Chart Overlay.
Delete the response_time overlay.
Add the Conversionoverlay.
Turn on View as Axis.
Click on Apply.
Drag this panel to the right of the second row.

You have completed the real-time version of the combo charts. It should look similar to the following screenshot:

Real-time combo charts with line overlays>

Real-time column charts with line overlays

It is time to build the second row of your real-time dashboard. Once again, we will use the cloning panel function:

Enter edit mode with Edit | Edit Panels.
Click Add Panel.
Clone the dynamic form-based dashboard: Hits vs Response Time:
Click Add to Dashboard.
Tip
Do not be surprised if the graph is not generated. Remember we are cloning from a form-based dashboard with input tokens.
Rename the panel to Traffic and Performance.

Change the Search String to remove input token references:

      SPL> index=main status_type="*" http_uri=* server_ip=* 
           | timechart count, avg(http_response_time) as response_time

Change the Time Range Scope to Explicit Selection.
Change the Time Range to Real-time and set its value as 24 Hours Ago.
Click on Save.
The chart will now populate the data.
Click on Done.

We will create another panel similar to the one we made previously. But this time, we will clone from the previous one to make our task easier.

Reload the browser. This is needed to load the newly-created panels in the clone selections.
Enter edit mode by Edit | Edit Panels.
Clone the Hits vs Response Time panel:
Rename the new panel to Booking Conversion.

Change the Search String:

     SPL> index=main http_uri=/booking/reservation OR 
          http_uri=/booking/confirmation 
          | timechart count by http_uri | rename /booking/confirmation AS 
          Confirmation, /booking/reservation AS Reservation 
          | eval Conversion=Reservation/Confirmation 
          | fields _time, Reservation, Confirmation, Conversion

Change Time Range to Real-time | 24 Hours Ago.
Click the Visualization Options dropdown.
Select the second option in Stack Mode as stacked.
Click Chart Overlay.
Delete the response_time overlay.
Add the Conversionoverlay.
Turn on View as Axis.
Click on Apply.
Drag this panel to the right of the second row.

You have completed the real-time version of the combo charts. It should look similar to the following screenshot:

Real-time combo charts with line overlays>

Creating a map called a choropleth

A choropleth, whose name comes from two Greek words meaning area/region and multitude, is a two-dimensional map where areas are designated by color shades or patterns to indicate the measured strength of a statistical indicator, such as sales per area or crime rates.

I'm sure you have already seen a choropleth, even if you didn't know what it was. Maybe you have seen areas of the US map shaded by state during a nationwide election. Or maybe you have seen a map of countries with a range of colors.

We cannot dig into the mathematical details of how a choropleth is created, but are fortunate that we can use Splunk 6.3+ to provide this effective visualization tool for us. We will create two choropleths to denote bookings by region and traffic by region.

Since we don't have a panel to clone from, we will create this from scratch:

Enter edit mode with Edit | Edit Panels.
Click on Add Panel.
Select New | Choropleth Map.
Change Time Range to Real-time and its value as 1 hour window.
In Content Title, type in Traffic Choropleth.
Type in this Search String, which includes a geomap command and makes use of one of the two geographic lookup maps that are included by default with Splunk. The one used here includes the United States; the other one is for the world. This geomap command asks for a map with the counts for different states. Shading is based on the relative magnitudes of the counts:
```
      SPL> index=main | iplocation client_ip | stats count by Region 
           | rename Region as featureId | geom geo_us_states
```
Click on Add to Dashboard.
Click the Visualization Options dropdown.
We will now put the United States in the center of the map and adjust the zoom level.
Change Latitude to 39.
Change Longitude to -98.
Change Zoom to 4.
Click on the Colors tab.
Change Number of Bins to 9. This will increase the color range by adding more gradient tones:
Click on Apply.
Click on Done.

Now reload your browser to allow this new panel to be added to the cloning panel selection.

Once you have cloned the Traffic Choropleth panel, change two things:

Title: Bookings Choropleth
Search String: index=main http_uri=/booking/confirmation http_status_code=200 | iplocation client_ip | stats count by Region | rename Region as featureId | geom geo_us_states

Now drag and position the second choropleth panel to the right of the other one to make the dashboard fluid.

You have now created a real-time single pane of glass dashboard. When you use this with your real production data, you can create a visualization that is useful and can produce all kinds of efficiencies. Your hard work can become a big hit!

Dashboard with advanced indicators, combo charts with line overlays, and choropleth charts

Summary

In this chapter, you have delved deeper into dashboard creation. You have learned about the different types of dashboard and when to use them. You have created a fully functional form-based dashboard that allowed you to change the inputs and affect the dashboard data. You have also learned how to use tokens and assign them to search panels, and how to create and modify advanced visualization tools. Finally, you have learned how to create a real-time dashboard with advanced visualization panels such as Single Value Panels with Trends and Choropleths map. In the next chapter, Chapter 7, Splunk SDK for JavaScript and D3.js, you will learn to use the Splunk System Development Kit to make your visualizations even more interesting.

Splunk: Enterprise Operational Intelligence Delivered

By : Derek Mock, Betsy Page Sigman, Paul R. Johnson, Erickson Delgado, Josh Diakun, Ashish Kumar Tulsiram Yadav

Splunk: Enterprise Operational Intelligence Delivered

By: Derek Mock, Betsy Page Sigman, Paul R. Johnson, Erickson Delgado, Josh Diakun, Ashish Kumar Tulsiram Yadav

Overview of this book

Related Content you might be interested in

Current Title:

Splunk: Enterprise Operational Intelligence Delivered

ChatGPT Prompts Book - Precision Prompts, Priming, Training & AI Writing Techniques for Mortals

GPT-3

Generative AI for Cloud Solutions

Chapter 6. Panes of Glass

Creating effective dashboards

Types of dashboard

Gathering information and business requirements

Dynamic form-based dashboard

Note

Creating a Status Distribution panel

Creating the Status Types Over Time panel

Creating the Hits vs Response Time panel

Arranging the dashboard

Panel options

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Column with line overlay combo chart - Hits vs Response Time

Gathering information and business requirements

Dynamic form-based dashboard

Note

Creating a Status Distribution panel

Creating the Status Types Over Time panel

Creating the Hits vs Response Time panel

Arranging the dashboard

Panel options

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Column with line overlay combo chart - Hits vs Response Time

Dynamic form-based dashboard

Note

Creating a Status Distribution panel

Creating the Status Types Over Time panel

Creating the Hits vs Response Time panel

Arranging the dashboard

Panel options

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Column with line overlay combo chart - Hits vs Response Time

Creating a Status Distribution panel

Creating the Status Types Over Time panel

Creating the Hits vs Response Time panel

Arranging the dashboard

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Creating the Status Types Over Time panel

Creating the Hits vs Response Time panel

Arranging the dashboard

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Creating the Hits vs Response Time panel

Arranging the dashboard

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Arranging the dashboard

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Panel options

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Column with line overlay combo chart - Hits vs Response Time

Pie chart - status distribution

Stacked area chart - Status Types Over Time

Stacked area chart - Status Types Over Time

Column with line overlay combo chart - Hits vs Response Time

Form inputs

Creating a time range input

Creating a radio input

Creating a dropdown input

Static Real-Time dashboard

Single Value Panels with color ranges

Creating panels by cloning

Single Value Panels with trends