Book Image

Mastering Elasticsearch 5.x - Third Edition

By : Bharvi Dixit
Book Image

Mastering Elasticsearch 5.x - Third Edition

By: Bharvi Dixit

Overview of this book

Elasticsearch is a modern, fast, distributed, scalable, fault tolerant, and open source search and analytics engine. Elasticsearch leverages the capabilities of Apache Lucene, and provides a new level of control over how you can index and search even huge sets of data. This book will give you a brief recap of the basics and also introduce you to the new features of Elasticsearch 5. We will guide you through the intermediate and advanced functionalities of Elasticsearch, such as querying, indexing, searching, and modifying data. We’ll also explore advanced concepts, including aggregation, index control, sharding, replication, and clustering. We’ll show you the modules of monitoring and administration available in Elasticsearch, and will also cover backup and recovery. You will get an understanding of how you can scale your Elasticsearch cluster to contextualize it and improve its performance. We’ll also show you how you can create your own analysis plugin in Elasticsearch. By the end of the book, you will have all the knowledge necessary to master Elasticsearch and put it to efficient use.
Table of Contents (13 chapters)

Introducing Logstash, Beats, and Kibana

We have already seen how to install and configure Elasticsearch. So, we are not going to repeat it again. We will proceed with learning three more components of Elastic Stack: Logstash, Beats, and Kibana.

Working with Logstash

Logstash is one of the most popular tools for collecting, parsing, and enriching log-based data (usually, data which has a timestamp associated with it) from multiple sources such as log files, databases, Twitter, Amazon S3, Amazon CloudWatch, Apache Kafka, and many others. After processing and transforming the data through Logstash, you can send it to either Elasticsearch or many other data stores such as MongoDB, Amazon S3, and so on.

Logstash architecture

Logstash has plugin-based architecture. As shown in the following figure, there are three components of Logstash: Input, Filter, and Output:

Logstash architecture

There are hundreds of input, filter, and output ready-made open source plugins available to be used and the best part is if you do not...