Book Image

Learning Elastic Stack 6.0

By : Pranav Shukla, Sharath Kumar M N
Book Image

Learning Elastic Stack 6.0

By: Pranav Shukla, Sharath Kumar M N

Overview of this book

The Elastic Stack is a powerful combination of tools for distributed search, analytics, logging, and visualization of data from medium to massive data sets. The newly released Elastic Stack 6.0 brings new features and capabilities that empower users to find unique, actionable insights through these techniques. This book will give you a fundamental understanding of what the stack is all about, and how to use it efficiently to build powerful real-time data processing applications. After a quick overview of the newly introduced features in Elastic Stack 6.0, you’ll learn how to set up the stack by installing the tools, and see their basic configurations. Then it shows you how to use Elasticsearch for distributed searching and analytics, along with Logstash for logging, and Kibana for data visualization. It also demonstrates the creation of custom plugins using Kibana and Beats. You’ll find out about Elastic X-Pack, a useful extension for effective security and monitoring. We also provide useful tips on how to use the Elastic Cloud and deploy the Elastic Stack in production environments. On completing this book, you’ll have a solid foundational knowledge of the basic Elastic Stack functionalities. You’ll also have a good understanding of the role of each component in the stack to solve different data processing problems.
Table of Contents (19 chapters)
Title Page
Credits
Disclaimer
About the Authors
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Data preparation


As Kibana is all about gaining insight from data, let's load some sample data that we will use as we follow the tutorial. One of the most common use cases is log analysis. For this tutorial, we will be loading Apache server logs into Elasticsearch using Logstash and then using it in Kibana for analysis/building visualizations.

https://github.com/elastic/elk-index-size-tests hosts a dump of Apache server logs that were collected for the site www.logstash.net for the period of May 2014 to June 2014. It contains 300,000 log events.

Navigate to https://github.com/elastic/elk-index-size-tests/blob/master/logs.gz and click the Download button. Unzip the logs.gz file.

Make sure you have Logstash version 5.6 and above installed. Create a config file named apache.conf in the $LOGSTASH_HOME\bin folder, as shown in the following code block:

input 
{ 
  file {
        path => "D:\Learnings\data\logs\logs"
        type => "logs"
        start_position => "beginning"
       }
}
...