The most common requirements of today standard applications are the search and analytics capabilities. On the market we can find a lot of solutions to answer to these need both in commercial and in open source world. One of the most used libraries for searching is Apache Lucene. This library is the base of a large number of search solutions such as Apache Solr, Indextank, and Elasticsearch.
Elasticsearch is one of the most powerful solution, written with the cloud and distributed computing in mind. Its main author, Shay Banon, famous for having developed Compass (http://www.compass-project.org), released the first version of Elasticsearch in March 2010.
Thus the main scope of Elasticsearch is to be a search engine; it also provides a lot of features that allows using it also as data-store and analytic engine via its aggregation framework.
Elasticsearch contains a lot of innovative features: JSON REST based, natively distributed in a map/reduce approach for both search and analytics, easy to set up and extensible with plugins. From 2010 when it started to be developed, to last version (5.x) there is a big evolution of the product becoming one of the most used datastore for a lot of markets. In this book we will go in depth on these changes and features and many others capabilities available in Elasticsearch.
Elasticsearch is also a product in continuous evolution and new functionalities are released both by the Elasticsearch Company (the company founded by Shay Banon to provide commercial support for Elasticsearch) and by Elasticsearch users as plugin (mainly available on GitHub). Today a lot of the major world players in IT industry (see some use cases at https://www.elastic.co/use-cases) are using Elasticsearch for its simplicity and advanced features.
In my opinion, Elasticsearch is probably one of the most powerful and easy-to-use search solution on the market. In writing this book and these recipes, I and the book reviewers have tried to transmit our knowledge, our passion, and best practices to better manage it.
Chapter 1, Getting Started, The goal of this chapter is to give the reader an overview of the basic concepts of Elasticsearch and the ways to communicate with it.
Chapter 2, Downloading and Setup, covers the basic steps to start using Elasticsearch from the simple install to a cloud ones.
Chapter 3, Managing Mappings, covers the correct definition of the data fields to improve both indexing and searching quality.
Chapter 4, Basic Operations, teaches the most common actions that are required to ingest data in Elasticsearch and to manage it.
Chapter 5, Search, talks about executing search, sorting and related API calls. The API discussed in this chapter are the main
Chapter 6, Text and Numeric Queries, talks about Search DSL part on text and numeric fields —the core of the search functionalities of Elasticsearch.
Chapter 7, Relationships and Geo Queries, talks about queries that works on related document (child/parent, nested) and geo located fields.
Chapter 8, Aggregations, covers another capability of Elasticsearch, the possibility to execute analytics on search results to improve both the user experience and to drill down the information contained in Elasticsearch.
Chapter 9, Scripting, shows how to customize Elasticsearch with scripting and use the scripting capabilities in different part of Elasticsearch (search, aggregation, and ingest) using different languages. The chapter is mainly focused on Painless the new scripting language developed by Elastic Team.
Chapter 10, Managing Clusters and Nodes, shows how to analyze the behavior of a cluster/node to understand common pitfalls.
Chapter 11, Backup and Restore, covers one of the most important component in managing data: Backup. It shows how to manage a distributed backup and restore of snapshots.
Chapter 12, User Interfaces, describes two of the most common user interfaces for Elasticsearch 5.x: Cerebro, mainly used for admin activities, and Kibana with X-Pack as a common UI extension for Elasticsearch.
Chapter 13, Ingest, talks about the new ingest functionality introduced in Elasticsearch 5.x to import data in Elasticsearch via an ingestion pipeline.
Chapter 14, Java Integration, describes how to integrate Elasticsearch in Java application using both REST and native protocols.
Chapter 15, Scala Integration, describes how to integrate Elasticsearch in Scala using elastic4s: an advanced type-safe and feature rich Scala library based on native Java API.
Chapter 16, Python Integration, covers the usage of the official Elasticsearch Python client.
Chapter 17, Plugin Development, describes how to create native plugins to extend Elasticsearch functionalities. Some examples show the plugin skeletons, the setup process, and their building.
Chapter 18, Big Data Integration, covers how to integrate Elasticsearch in common big data tools such as Apache Spark and Apache Pig.
For this book you will need a computer, of course. In terms of software required you don’t have to be worried, all the components we use are open source and available for every platform.
For all the REST example, the CURL software (http://curl.haxx.se/) is used to simulate the command from command line. It’s common preinstalled in Linux and Mac OS X operative systems. For Windows, it can be downloaded from its site and put in a PATH that can be called from command-line.
For the Chapter 14, Java Integration and and Chapter 17, Plugin Development, it is required the Maven build tool (http://maven.apache.org/), which is a standard for managing build, packaging and deploy in Java. It is natively supported in Java IDEs such as Eclipse and Intellij IDEA.
For Chapter 15, Scala Integration, SBT, (http://www.scala-sbt.org/) is required to compile Scala projects, but it can be also used with IDE that supports Scala such as Eclipse and Intellij IDEA.
The Chapter 16, Python Integration, requires the Python interpreter installed. By default it’s available on Linux and Mac OS X , for Windows can be downloaded from the official python site (http://www.python.org). For the current examples the version 2.X is used.
This book is for developers who want to start using both Elasticsearch and at the same time improve their Elasticsearch knowledge. The book covers all the aspects of using Elasticsearch and provides solutions and hints for everyday usage. The recipes are reduced in complexity to easy focus the reader on the discussed Elasticsearch aspect and to easily memorize the Elasticsearch functionalities.
The latter chapters that discuss the Elasticsearch integration in JAVA, Scala, Python, and Big Data tools show the user how to integrate the power of Elasticsearch in their applications.
The chapter, that talks about plugin development, shows an advanced usage of Elasticsearch and its core extension, so some skilled Java know-how is required.
In this book, you will find several headings that appear frequently (Getting ready, How to do it, How it works, There's more, and See also).
To give clear instructions on how to complete a recipe, we use these sections as follows:
This section tells you what to expect in the recipe, and describes how to set up any software or any preliminary settings required for the recipe.
This section usually consists of a detailed explanation of what happened in the previous section.
This section consists of additional information about the recipe in order to make the reader more knowledgeable about the recipe.
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.
Code words in text are shown as follows: "After the name and the type parameters, usually a river requires an extra configuration that can be passed in the _metaproperty "
A block of code is set as follows:
cluster.name: elasticsearch node.name: "My wonderful server" network.host: 192.168.0.1 discovery.zen.ping.unicast.hosts: ["192.168.0.2","192.168.0.3[9300-9400]"]
Any command-line input or output is written as follows:
curl -XDELETE 'http://127.0.0.1:9200/_river/my_river/'
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.
To send us general feedback, simply send an e-mail to [email protected], and mention the book title via the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on http://www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
- Log in or register to our website using your e-mail address and password.
- Hover the mouse pointer on the SUPPORT tab at the top.
- Click on Code Downloads & Errata.
- Enter the name of the book in the Search box.
- Select the book for which you're looking to download the code files.
- Choose from the drop-down menu where you purchased this book from.
- Click on Code Download.
You can also download the code files by clicking on the Code Files button on the book's webpage at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
- WinRAR / 7-Zip for Windows
- Zipeg / iZip / UnRarX for Mac
- 7-Zip / PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Elasticsearch-5x-Cookbook-Third-Edition. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.
Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at [email protected] with a link to the suspected pirated material.
We appreciate your help in protecting our authors, and our ability to bring you valuable content.
You can contact us at [email protected] if you are having a problem with any aspect of the book, and we will do our best