Book Image

ElasticSearch Cookbook

By : Alberto Paro
Book Image

ElasticSearch Cookbook

By: Alberto Paro

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
ElasticSearch Cookbook
Alberto Paro
Jan, 2015 | 472 pages
No titles found
Table of Contents (20 chapters)
ElasticSearch Cookbook Second Edition
ElasticSearch Cookbook Second Edition
Credits
Credits
About the Author
About the Author
Acknowledgments
Acknowledgments
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Getting Started
Getting Started
Introduction
Understanding nodes and clusters
Understanding node services
Managing your data
Understanding clusters, replication, and sharding
Communicating with ElasticSearch
Using the HTTP protocol
Using the native protocol
Using the Thrift protocol
2
Downloading and Setting Up
Downloading and Setting Up
Introduction
Downloading and installing ElasticSearch
Setting up networking
Setting up a node
Setting up for Linux systems
Setting up different node types
Installing plugins in ElasticSearch
Installing a plugin manually
Removing a plugin
Changing logging settings
3
Managing Mapping
Managing Mapping
Introduction
Using explicit mapping creation
Mapping base types
Mapping arrays
Mapping an object
Mapping a document
Using dynamic templates in document mapping
Managing nested objects
Managing a child document
Adding a field with multiple mappings
Mapping a geo point field
Mapping a geo shape field
Mapping an IP field
Mapping an attachment field
Adding metadata to a mapping
Specifying a different analyzer
Mapping a completion suggester
4
Basic Operations
Basic Operations
Introduction
Creating an index
Deleting an index
Opening/closing an index
Putting a mapping in an index
Getting a mapping
Deleting a mapping
Refreshing an index
Flushing an index
Optimizing an index
Checking if an index or type exists
Managing index settings
Using index aliases
Indexing a document
Getting a document
Deleting a document
Updating a document
Speeding up atomic operations (bulk operations)
Speeding up GET operations (multi GET)
5
Search, Queries, and Filters
Search, Queries, and Filters
Introduction
Executing a search
Sorting results
Highlighting results
Executing a scan query
Suggesting a correct query
Counting matched results
Deleting by query
Matching all the documents
Querying/filtering for a single term
Querying/filtering for multiple terms
Using a prefix query/filter
Using a Boolean query/filter
Using a range query/filter
Using span queries
Using a match query
Using an ID query/filter
Using a has_child query/filter
Using a top_children query
Using a has_parent query/filter
Using a regexp query/filter
Using a function score query
Using exists and missing filters
Using and/or/not filters
Using a geo bounding box filter
Using a geo polygon filter
Using geo distance filter
Using a QueryString query
Using a template query
6
Aggregations
Aggregations
Introduction
Executing an aggregation
Executing the stats aggregation
Executing the terms aggregation
Executing the range aggregation
Executing the histogram aggregation
Executing the date histogram aggregation
Executing the filter aggregation
Executing the global aggregation
Executing the geo distance aggregation
Executing nested aggregation
Executing the top hit aggregation
7
Scripting
Scripting
Introduction
Installing additional script plugins
Managing scripts
Sorting data using script
Computing return fields with scripting
Filtering a search via scripting
Updating a document using scripts
8
Rivers
Rivers
Introduction
Managing a river
Using the CouchDB river
Using the MongoDB river
Using the RabbitMQ river
Using the JDBC river
Using the Twitter river
9
Cluster and Node Monitoring
Cluster and Node Monitoring
Introduction
Controlling cluster health via the API
Controlling cluster state via the API
Getting cluster node information via the API
Getting node statistics via the API
Managing repositories
Executing a snapshot
Restoring a snapshot
Installing and using BigDesk
Installing and using ElasticSearch Head
Installing and using SemaText SPM
Installing and using Marvel
10
Java Integration
Java Integration
Introduction
Creating an HTTP client
Creating a native client
Managing indices with the native client
Managing mappings
Managing documents
Managing bulk actions
Building a query
Executing a standard search
Executing a search with aggregations
Executing a scroll/scan search
11
Python Integration
Python Integration
Introduction
Creating a client
Managing indices
Managing mappings
Managing documents
Executing a standard search
Executing a search with aggregations
12
Plugin Development
Plugin Development
Introduction
Creating a site plugin
Creating a native plugin
Creating a REST plugin
Creating a cluster action
Creating an analyzer plugin
Creating a river plugin
Index
Index

Using the native protocol

ElasticSearch provides a native protocol, used mainly for low-level communication between nodes, but very useful for fast importing of huge data blocks. This protocol is available only for Java Virtual Machine (JVM) languages and commonly is used in Java, Groovy, and Scala.

Getting ready

You need a working instance of the ElasticSearch cluster; the standard port number for native protocol is 9300.

How to do it...

The following are the steps required to use the native protocol in a Java environment (we'll discuss this in depth in Chapter 10, Java Integration):

  1. Before starting, we must be sure that Maven loads the Elasticsearch.jar file by adding the following code to the pom.xml file:

    <dependency>
  <groupId>org.elasticsearch</groupId>
  <artifactId>elasticsearch</artifactId>
  <version>1.4.1</version>
</dependency>

  2. Depending on the Elasticsearch.jar file, creating a Java client is quite easy:

    import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
…
Settings settings = ImmutableSettings.settingsBuilder()
.put("client.transport.sniff", true).build();
  // we define a new settings
  // using sniff transport allows to autodetect other nodes
Client client = new TransportClient(settings)
  .addTransportAddress(new InetSocketTransportAddress("127.0.0.1", 9300));
  // a client is created with the settings

How it works...

To initialize a native client, a settings object is required, which contains some configuration parameters. The most important ones are:

  • cluster.name: This is the name of the cluster

  • client.transport.sniff: This allows you to sniff out the rest of the cluster and add them into its list of machines to use

With the settings object, it's possible to initialize a new client by giving an IP address and port a number (default 9300).

There's more...

The native protocol is the internal one used in ElasticSearch. It's the fastest protocol that is available to communicate with ElasticSearch.

The native protocol is an optimized binary and works only for JVM languages; to use this protocol, you need to include the elasticsearch.jar in your JVM project. Because it depends on ElasticSearch implementation, it must be the same version of ElasticSearch cluster.

For this reason, every time you update ElasticSearch, you need to update the elasticsearch.jar file on which it depends and if there are internal API changes, you need to update your code.

To use this protocol, you need to study the internals of ElasticSearch, so it's not as easy to use as HTTP and Thrift protocol.

Native protocol is useful for massive data import. But as ElasticSearch is mainly thought as a REST HTTP server to communicate with, it lacks support for everything that is not standard in the ElasticSearch core, such as the plugin's entry points. So using this protocol, you are unable to call entry points made by external plugins.

Note

The native protocol seems the most easy to integrate in a Java/JVM project. However, due to its nature that follows the fast release cycles of ElasticSearch, it changes very often. Also, for minor release upgrades, your code is more likely to be broken. Thus, ElasticSearch developers wisely tries to fix them in the latest releases.

See also

Previous Section
End of Section 9
Next Section