If you want to follow the examples that appear in the next few chapters, install the ImplementingSplunkDataGenerator demo app by following these steps:

This data generator app will produce about 16 megabytes of output per day. The app can be disabled so that it stops producing data by using Manage apps... under the App menu.

Within the Search & Reporting app, the user is presented with the Summary view, which contains information about the data that the user searches by default. This is an important distinction; in a mature Splunk installation, not all users will always search all data by default. But if this is your first trip into Search & Reporting, you'll see the following:

From the screen depicted in the previous screenshot, you can access the Splunk documentation related to What to Search and How to Search. Once you have at least some data indexed (a topic we'll discuss later), Splunk will provide some statistics on the available data under What to Search.

What to Search is shown in the following screenshot:

In previous versions of Splunk, panels such as the All indexed data panel provided statistics for a user's indexed data. Other panels gave a breakdown of data using three important pieces of metadata—Source, Sourcetype, and Hosts. In the current version, 7.0.0, you access this information by clicking on the button labeled Data Summary, which presents the following to the user:

This dialog splits the information into three tabs—Hosts, Sources and Sourcetypes:

Let's move on now and discuss each of the Splunk widgets (just below the app name). The first widget is the navigation bar:

As a general rule, within Splunk, items with downward triangles are menus. Items without a downward triangle are links.

We will cover customizing the navigation bar in Chapter 8, Working with Apps.

Next, we find the Search bar. This is where the magic starts. We'll go into great detail shortly:

Okay, we've finally made it to search. This is where the real power of Splunk lies.

For our first search, we will search for the word (not case-specific) error. Click in the search bar, type the word error, and then either press Enter or click on the magnifying glass to the right of the bar:

Upon initiating the search, we are taken to the search results page (which hasn't really changed in version 7.0):

Refer to the Using the time picker section for details on changing the time frame of your search.

Let's inspect the elements on this page. Below the Search bar, we have the event count, action icons, and menus:

Starting from the left, we have the following:

Now we'll skip to the timeline below the action icons:

Along with providing a quick overview of the event distribution over a period of time, the timeline is also a very useful tool for selecting sections of time. Placing the pointer over the timeline displays a popup for the number of events in that slice of time. Clicking on the timeline selects the events for a particular slice of time.

Clicking and dragging selects a range of time:

Once you have selected a period of time, clicking on Zoom to selection changes the time frame and reruns the search for that specific slice of time. Repeating this process is an effective way to drill down to specific events.

Deselect shows all events for the time range selected in the time picker.

Zoom out changes the window of time to a larger period around the events in the current time frame.

To the left of the search results, we find the field picker. This is a great tool for discovering patterns and filtering search results:

Fields

The field list contains two lists.

• Selected Fields, which have their values displayed under the search event in the search results
• Interesting Fields, which are other fields that Splunk has picked out for you

Above the field list are two links, Hide Fields and All Fields:

• Hide Fields: Hides the field list area from the view
• All Fields: Takes you to the Selected Fields window:

We are almost through with all the widgets on the page. We still have a number of items to cover in the search results section, though, just to be thorough:

As you can see in the previous screenshot, at the top of this section, we have the number of events displayed. When viewing all results in their raw form, this number will match the number above the timeline. This value can be changed either by making a selection on the timeline or by using other search commands.

Next, we have the action icons (described earlier) that affect these particular results.

Under the action icons, we have four results tabs:

Under the previously described tabs, is the timeline that we will cover in more detail later in this chapter.

Events viewer

Finally, we make it to the actual events. Let's examine a single event:

Starting from the left, we have:

• Event Details: Clicking here (indicated by the right facing arrow) opens the selected event, provides specific information about the event by type, field, and value, and allows you the ability to perform specific actions on a particular event field. In addition, Splunk offers a button labeled Event Actions to access workflow actions, a few of which are always available.
• Build Event Type: Event types are a way to name events that match a certain query. We will dive into event types in Chapter 7, Extending Search.
• Extract Fields: This launches an interface for creating custom field extractions. We will cover field extraction in Chapter 3, Tables, Charts, and Fields.
• Show Source: This pops up a window with a simulated view of the original source.
• The event number: Raw search results are always returned in the order most recent first.
• Next appear any workflow actions that have been configured. Workflow actions let you create new searches or links to other sites, using data from an event. We will discuss workflow actions in Chapter 7, Extending Search.
• Next comes the parsed date from this event, displayed in the time zone selected by the user. This is an important and often confusing distinction. In most installations, everything is in one time zone—the servers, the user, and the events. When one of these three things is not in the same time zone as the others, things can get confusing. We will discuss time in great detail in Chapter 2, Understanding Search.
• Next, we see the raw event itself. This is what Splunk saw as an event. With no help, Splunk can do a good job finding the date and breaking lines appropriately; but as we will see later, with a little help, event parsing can be more reliable and more efficient.
• Below the event are the fields that were selected in the field picker. Clicking on the value adds the field value to the search.