Even structured data is useless if it can’t help you to take strategic decisions and improve existing system. If you love to play with data, or your job requires you to process custom log formats, design a scalable analysis system, and manage logs to do real-time data analysis, this book is your one-stop solution. By combining the massively popular Elasticsearch, Logstash, Beats and Kibana, ELK Stack has advanced to Elastic Stack that delivers actionable insights in near real time from almost any type of structured or unstructured data.
This book brushes up your basic knowledge of implementing the Elastic Stack and then dives deeper into complex and advanced scenarios. We’ll help you with data analytics challenges and take you through practical scenario of an intranet portal to understand utilization of Elastic Stack components. You will be able to grasp advanced techniques for log analysis and visualization. Newly announced features such as Beats and X-Pack are also covered in detail with examples.
Toward the end, you will see how to use the Elastic stack for real-world case studies and we’ll show you some best practices and troubleshooting techniques for the Elastic Stack.
Chapter 1, Elastic Stack Overview, covers the shift from ELK Stack to Elastic Stack followed by setup of various components of Elastic Stack.
Chapter 2, Stepping into Elasticsearch, takes us to how Elasticsearch started as a project, how Elasticsearch works and covering various Elasticsearch API’s and Aggregations.
Chapter 3, Exploring Logstash and Its Plugins, covers introduction of Logstash along with understanding it’s architecture. It also covers the various plugins with suitable examples. At the end, a Logstash configuration file is shown for parsing logs.
Chapter 4, Kibana Interface, teaches about the various interfaces present in Kibana in depth along with an example to demonstrate how to combine all the interfaces to create a dashboard.
Chapter 5, Using Beats, takes us to introducing the beats, understanding how beat differs from Logstash followed by exploring various beats, their functionalities and setup steps. At the end, we explored how to use Beats in Elastic Stack.
Chapter 6, Elastic Stack in Action, covers a real-world use-case of an Intranet Portal server and showcases and how to use Elastic Stack components to solve the problem.
Chapter 7, Customizing Elastic Stack, teaches us how to extend each component of Elastic Stack and how to create a plugin for our use-cases.
Chapter 8, Elasticsearch APIs, takes us to various Elasticsearch API’s along with understanding Elasticsearch modules, Ingest nodes, Discovery pPlugins and how to use Java client to access various Elasticsearch operations.
Chapter 9, X-Pack: Security and Monitoring, covers introduction of X-Pack along with installation of X-Pack. It also covers the usage and functionalities provided by Shield, Marvel and Profiler.
Chapter 10, X-Pack: Alerting, Graph, and Reporting, teaches us about the usage and functionalities of Watcher, Graph and Reporting features.
Chapter 11, Best Practices, takes us to understand why do we need to follow best practices along with listing of various best practices which should be followed which has been categorized into multiple sub-sections.
Chapter 12, Case Study-Meetup, covers complete coverage of understanding the problem statement followed by extending Logstash and creating a plugin to fetch required information. It then takes us to understand how to utilize Elastic Stack components to cover end-to-end understanding of Meetup data and showcasing the powerful capabilities of Elastic Stack for data analytics.
Following table lists all required software and tools needed to execute example in the book. Wherever requires, links to download the software is also present within the chapter as well.
|
Software |
Version |
Link |
|
Elasticsearch |
5.1.1 |
https://www.elastic.co/downloads/past-releases/elasticsearch-5-1-1 |
|
Logstash |
5.1.1 |
https://www.elastic.co/downloads/past-releases/logstash-5-1-1 |
|
Kibana |
5.1.1 | |
|
Filebeat |
5.1.1 |
https://www.elastic.co/downloads/past-releases/filebeat-5-1-1 |
|
Packetbeat |
5.1.1 |
https://www.elastic.co/downloads/past-releases/packetbeat-5-1-1 |
|
Winlogbeat |
5.1.1 |
https://www.elastic.co/downloads/past-releases/winlogbeat-5-1-1 |
|
Metricbeat |
5.1.1 |
https://www.elastic.co/downloads/past-releases/metricbeat-5-1-1 |
|
Elasticsearch |
1.4.0 |
https://www.elastic.co/downloads/past-releases/elasticsearch-1-4-0 |
|
Liferay |
6.2CEGA4 | |
|
Java |
8.x |
http://www.oracle.com/technetwork/java/javase/downloads/index.html |
|
Elasticray |
1.2.0 |
https://web.liferay.com/marketplace/-/mp/application/41044606 |
|
Go |
1.7.5 | |
|
Ruby |
2.4.0 | |
|
NodeJS |
6.9.0 | |
|
Gradle |
2.13 | |
|
Python |
2.7.10 | |
|
Virtualenv | ||
|
cookiecutter |
If you have heard the word ELK stack and want to learn more about it’s latest development and how it became Elastic Stack, this book is for you. If you use analytics or like to play with visualizations on your data, this book helps you to understand how the components of the stack can help you.
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "The next lines of code read the link and assign it to the to the BeautifulSoup function."
A block of code is set as follows:
#import packages into the project from bs4 import BeautifulSoup from urllib.request import urlopen import pandas as pd
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
<head>
<script src="d3.js" charset="utf-8"></script>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width">
<title>JS Bin</title>
</head>Any command-line input or output is written as follows:
C:\Python34\Scripts> pip install -upgrade pip C:\Python34\Scripts> pip install pandas
New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "In order to download new modules, we will go to Files | Settings | Project Name | Project Interpreter."
Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail [email protected], and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
- Log in or register to our website using your e-mail address and password.
- Hover the mouse pointer on the
SUPPORTtab at the top. - Click on
Code Downloads & Errata. - Enter the name of the book in the
Searchbox. - Select the book for which you're looking to download the code files.
- Choose from the drop-down menu where you purchased this book from.
- Click on
Code Download.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
- WinRAR / 7-Zip for Windows
- Zipeg / iZip / UnRarX for Mac
- 7-Zip / PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Mastering-Elastic-Stack. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at [email protected] with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.