One of the main requirements of today's applications is search capability. In the market, we can find a lot of solutions that answer this need, both in commercial as well as the open source world. One of the most used libraries for searching is Apache Lucene. This library is the base of a large number of search solutions such as Apache Solr, Indextank, and ElasticSearch.
ElasticSearch is written with both cloud and distributed computing in mind. Its main author, Shay Banon, who is famous for having developed Compass (http://www.compass-project.org), released the first version of ElasticSearch in March 2010.
Thus, the main scope of ElasticSearch is to be a search engine; it also provides a lot of features that allow you to use it as a data store and an analytic engine using aggregations.
ElasticSearch contains a lot of innovative features: it is JSON/REST-based, natively distributed in a Map/Reduce approach, easy to set up, and extensible with plugins. In this book, we will go into the details of these features and many others available in ElasticSearch.
Before ElasticSearch, only Apache Solr was able to provide some of these functionalities, but it was not designed for the cloud and does not use the JSON/REST API. In the last few years, this situation has changed a bit with the release of the SolrCloud in 2012. For users who want to more thoroughly compare these two products, I suggest you read posts by Rafał Kuć, available at http://blog.sematext.com/2012/08/23/solr-vs-elasticsearch-part-1-overview/.
ElasticSearch is a product that is in a state of continuous evolution, and new functionalities are released by both the ElasticSearch company (the company founded by Shay Banon to provide commercial support for ElasticSearch) and ElasticSearch users as plugins (mainly available on GitHub).
Founded in 2012, the ElasticSearch company has raised a total of USD 104 million in funding. ElasticSearch's success can best be described by the words of Steven Schuurman, the company's cofounder and CEO:
It's incredible to receive this kind of support from our investors over such a short period of time. This speaks to the importance of what we're doing: businesses are generating more and more data—both user- and machine-generated—and it has become a strategic imperative for them to get value out of these assets, whether they are starting a new data-focused project or trying to leverage their current Hadoop or other Big data investments.
ElasticSearch has an impressive track record for its search product, powering customers such as Fourquare (which indexes over 50 million venues), the online music distribution platform SoundCloud, StumbleUpon, and the enterprise social network Xing, which has 14 million members. It also powers GitHub, which searches 20 terabytes of data and 1.3 billion files, and Loggly, which uses ElasticSearch as a key value store to index clusters of data for rapid analytics of logfiles.
In my opinion, ElasticSearch is probably one of the most powerful and easy-to-use search solutions on the market. Throughout this book and these recipes, the book's reviewers and I have sought to transmit our knowledge, passion, and best practices to help readers better manage ElasticSearch.
Chapter 1, Getting Started, gives you an overview of the basic concepts of ElasticSearch and the ways to communicate with it.
Chapter 2, Downloading and Setting Up, shows the basic steps to start using ElasticSearch, from the simple installation to running multiple nodes.
Chapter 3, Managing Mapping, covers the correct definition of data fields to improve both the indexing and search quality.
Chapter 4, Basic Operations, shows you the common operations that are required to both ingest and manage data in ElasticSearch.
Chapter 5, Search, Queries, and Filters, covers the core search functionalities in ElasticSearch. The search DSL is the only way to execute queries in ElasticSearch.
Chapter 6, Aggregations, covers another capability of ElasticSearch: the possibility to execute analytics on search results in order to improve the user experience and drill down the information.
Chapter 7, Scripting, shows you how to customize ElasticSearch with scripting in different programming languages.
Chapter 8, Rivers, extends ElasticSearch to give you the ability to pull data from different sources such as databases, NoSQL solutions, and data streams.
Chapter 9, Cluster and Node Monitoring, shows you how to analyze the behavior of a cluster/node to understand common pitfalls.
Chapter 10, Java Integration, describes how to integrate ElasticSearch in a Java application using both REST and native protocols.
Chapter 11, Python Integration, covers the usage of the official ElasticSearch Python client and the Pythonic PyES library.
Chapter 12, Plugin Development, describes how to create the different types of plugins: site and native plugins. Some examples show the plugin skeletons, the setup process, and their build.
For this book, you will need a computer running a Windows OS, Macintosh OS, or Linux distribution. In terms of the additional software required, you don't have to worry, as all the components you will need are open source and available for every major OS platform.
For all the REST examples, the cURL software (http://curl.haxx.se/) will be used to simulate the command from the command line. It comes preinstalled on Linux and Mac OS X operating systems. For Windows, it can be downloaded from its site and added in a PATH that can be called from the command line.
Chapter 10, Java Integration, and Chapter 12, Plugin Development, require the Maven build tool (http://maven.apache.org/), which is a standard tool to manage builds, packaging, and deploying in Java. It is natively supported on most of the Java IDEs, such as Eclipse and IntelliJ IDEA.
Chapter 11, Python Integration, requires the Python Interpreter installed on your computer. It's available on Linux and Mac OS X by default. For Windows, it can be downloaded from the official Python website (http://www.python.org). The examples in this chapter have been tested using version 2.x.
This book is for developers and users who want to begin using ElasticSearch or want to improve their knowledge of ElasticSearch. This book covers all the aspects of using ElasticSearch and provides solutions and hints for everyday usage. The recipes have reduced complexity so it is easy for readers to focus on the discussed ElasticSearch aspect and easily and fully understand the ElasticSearch functionalities.
The chapters toward the end of the book discuss ElasticSearch integration with Java and Python programming languages; this shows the users how to integrate the power of ElasticSearch into their Java- and Python-based applications.
Chapter 12, Plugin Development, talks about the advanced use of ElasticSearch and its core extensions, so you will need some prior Java knowledge to understand this chapter fully.
This book contains the following sections:
This section tells us what to expect in the recipe, and describes how to set up any software or any preliminary settings needed for the recipe.
This section usually consists of a brief and detailed explanation of what happened in the previous section.
It consists of additional information about the recipe in order to make the reader more anxious about the recipe.
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "After the name and type parameters, usually a river requires an extra configuration that can be passed in the _meta property."
A block of code is set as follows:
cluster.name: elasticsearch node.name: "My wonderful server" network.host: 192.168.0.1 discovery.zen.ping.unicast.hosts: ["192.168.0.2","192.168.0.3[9300-9400]"]
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
cluster.name: elasticsearch
node.name: "My wonderful server"
network.host: 192.168.0.1
discovery.zen.ping.unicast.hosts: ["192.168.0.2","192.168.0.3[9300-9400]"]Any command-line input or output is written as follows:
curl -XDELETE 'http://127.0.0.1:9200/_river/my_river/'
New terms and important words are shown in bold. Words you see on the screen, in menus or dialog boxes, for example, appear in the text like this: "If you don't see the cluster statistics, put your node address to the left and click on the connect button."
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles you really get the most out of.
To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.
If there is a topic you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you get the most from your purchase.
You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you. The code bundle is also available on GitHub at https://github.com/aparo/elasticsearch-cookbook-second-edition.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors, and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.