Book Image

Mastering Elasticsearch 5.x - Third Edition

Book Image

Mastering Elasticsearch 5.x - Third Edition

Overview of this book

Elasticsearch is a modern, fast, distributed, scalable, fault tolerant, and open source search and analytics engine. Elasticsearch leverages the capabilities of Apache Lucene, and provides a new level of control over how you can index and search even huge sets of data. This book will give you a brief recap of the basics and also introduce you to the new features of Elasticsearch 5. We will guide you through the intermediate and advanced functionalities of Elasticsearch, such as querying, indexing, searching, and modifying data. We’ll also explore advanced concepts, including aggregation, index control, sharding, replication, and clustering. We’ll show you the modules of monitoring and administration available in Elasticsearch, and will also cover backup and recovery. You will get an understanding of how you can scale your Elasticsearch cluster to contextualize it and improve its performance. We’ll also show you how you can create your own analysis plugin in Elasticsearch. By the end of the book, you will have all the knowledge necessary to master Elasticsearch and put it to efficient use.
Table of Contents (20 chapters)
Mastering Elasticsearch 5.x - Third Edition
Credits
About the Author
Acknowledgements
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Managing time-based indices efficiently using shrink and rollover APIs


Recently, we talked a lot about how to scale Elasticsearch clusters and some general guidelines to follow while going into production. In this section, we are going to talk about two new APIs introduced in Elasticsearch 5.0. The Shrink and Rollover APIs. Both of these APIs are specially designed for managing time series-based indices such as, daily-/weekly-/monthly-created indices for logs, or an index for each week or month of tweets.

We know these basic points related to shards of an index:

  • We need to define the number of shards in advance at the time of index creation and we can't increase or decrease the number of shards for index once it is created.

  • The greater the number of shards, the more indexing throughput, the lesser the search speed, and greater number of resources are needed.

Both of these problems may be an overkill for the performance and management of your cluster when the data size grows and scaling is needed...