Book Image

Elasticsearch 5.x Cookbook - Third Edition

By : Alberto Paro
Book Image

Elasticsearch 5.x Cookbook - Third Edition

By: Alberto Paro

Overview of this book

Elasticsearch is a Lucene-based distributed search server that allows users to index and search unstructured content with petabytes of data. This book is your one-stop guide to master the complete Elasticsearch ecosystem. We’ll guide you through comprehensive recipes on what’s new in Elasticsearch 5.x, showing you how to create complex queries and analytics, and perform index mapping, aggregation, and scripting. Further on, you will explore the modules of Cluster and Node monitoring and see ways to back up and restore a snapshot of an index. You will understand how to install Kibana to monitor a cluster and also to extend Kibana for plugins. Finally, you will also see how you can integrate your Java, Scala, Python, and Big Data applications such as Apache Spark and Pig with Elasticsearch, and add enhanced functionalities with custom plugins. By the end of this book, you will have an in-depth knowledge of the implementation of the Elasticsearch architecture and will be able to manage data efficiently and effectively with Elasticsearch.

Related Content you might be interested in

Current Title:

Small book image
Elasticsearch 5.x Cookbook - Third Edition
Alberto Paro
Feb, 2017 | 696 pages
Small book image
Mastering Elasticsearch 5.x
Bharvi Dixit
Feb, 2017 | 428 pages
Small book image
Learning Elasticsearch
Abhishek Andhavarapu
Jun, 2017 | 404 pages
Small book image
Elasticsearch 7 Quick Start Guide
Douglas Miller | Anurag Srivastava
Oct, 2019 | 186 pages
Table of Contents (25 chapters)
Credits
Credits
About the Author
About the Author
About the Reviewer
About the Reviewer
www.PacktPub.com
www.PacktPub.com
Customer Feedback
Customer Feedback
Dedication
Dedication
Preface
Preface
Free Chapter
1
Getting Started
Getting Started
Introduction
Understanding node and cluster
Understanding node services
Managing your data
Understanding cluster, replication, and sharding
Communicating with Elasticsearch
Using the HTTP protocol
Using the native protocol
2
Downloading and Setup
Downloading and Setup
Introduction
Downloading and installing Elasticsearch
Setting up networking
Setting up a node
Setting up for Linux systems
Setting up different node types
Setting up a client node
Setting up an ingestion node
Installing plugins in Elasticsearch
Installing plugins manually
Removing a plugin
Changing logging settings
Setting up a node via Docker
3
Managing Mappings
Managing Mappings
Introduction
Using explicit mapping creation
Mapping base types
Mapping arrays
Mapping an object
Mapping a document
Using dynamic templates in document mapping
Managing nested objects
Managing child document
Adding a field with multiple mapping
Mapping a GeoPoint field
Mapping a GeoShape field
Mapping an IP field
Mapping an attachment field
Adding metadata to a mapping
Specifying a different analyzer
Mapping a completion field
4
Basic Operations
Basic Operations
Introduction
Creating an index
Deleting an index
Opening/closing an index
Putting a mapping in an index
Getting a mapping
Reindexing an index
See also
Refreshing an index
Flushing an index
ForceMerge an index
Shrinking an index
Checking if an index or type exists
Managing index settings
Using index aliases
Rollover an index
Indexing a document
Getting a document
Deleting a document
Updating a document
Speeding up atomic operations (bulk operations)
Speeding up GET operations (multi GET)
5
Search
Search
Introduction
Executing a search
Sorting results
Highlighting results
Executing a scrolling query
Using the search_after functionality
Returning inner hits in results
Suggesting a correct query
Counting matched results
Explaining a query
Query profiling
Deleting by query
Updating by query
Matching all the documents
Using a boolean query
6
Text and Numeric Queries
Text and Numeric Queries
Introduction
Using a term query
Using a terms query
Using a prefix query
Using a wildcard query
Using a regexp query
Using span queries
Using a match query
Using a query string query
Using a simple query string query
Using the range query
The common terms query
Using IDs query
Using the function score query
Using the exists query
Using the template query
7
Relationships and Geo Queries
Relationships and Geo Queries
Introduction
Using the has_child query
Using the has_parent query
Using nested queries
Using the geo_bounding_box query
Using the geo_polygon query
Using the geo_distance query
Using the geo_distance_range query
8
Aggregations
Aggregations
Introduction
Executing an aggregation
Executing stats aggregations
Executing terms aggregation
Executing significant terms aggregation
Executing range aggregations
Executing histogram aggregations
Executing date histogram aggregations
Executing filter aggregations
Executing filters aggregations
Executing global aggregations
Executing geo distance aggregations
Executing children aggregations
Executing nested aggregations
Executing top hit aggregations
Executing a matrix stats aggregation
Executing geo bounds aggregations
Executing geo centroid aggregations
9
Scripting
Scripting
Introduction
Painless scripting
Installing additional script plugins
Managing scripts
Sorting data using scripts
Computing return fields with scripting
Filtering a search via scripting
Using scripting in aggregations
Updating a document using scripts
Reindexing with a script
10
Managing Clusters and Nodes
Managing Clusters and Nodes
Introduction
Controlling cluster health via an API
Controlling cluster state via an API
Getting nodes information via API
Getting node statistics via the API
Using the task management API
Hot thread API
Managing the shard allocation
Monitoring segments with the segment API
Cleaning the cache
11
Backup and Restore
Backup and Restore
Introduction
Managing repositories
Executing a snapshot
Restoring a snapshot
Setting up a NFS share for backup
Reindexing from a remote cluster
12
User Interfaces
User Interfaces
Introduction
Installing and using Cerebro
Installing Kibana and X-Pack
Managing Kibana dashboards
Monitoring with Kibana
Using Kibana dev-console
Visualizing data with Kibana
Installing Kibana plugins
Generating graph with Kibana
13
Ingest
Ingest
Introduction
Pipeline definition
Put an ingest pipeline
Get an ingest pipeline
Delete an ingest pipeline
Simulate an ingest pipeline
Built-in processors
Grok processor
Using the ingest attachment plugin
Using the ingest GeoIP plugin
14
Java Integration
Java Integration
Introduction
Creating a standard Java HTTP client
Creating an HTTP Elasticsearch client
Creating a native client
Managing indices with the native client
Managing mappings
Managing documents
Managing bulk actions
Building a query
Executing a standard search
Executing a search with aggregations
Executing a scroll search
15
Scala Integration
Scala Integration
Introduction
Creating a client in Scala
Managing indices
Managing mappings
Managing documents
Executing a standard search
Executing a search with aggregations
16
Python Integration
Python Integration
Introduction
Creating a client
Managing indices
Managing mappings include the mapping
Managing documents
Executing a standard search
Executing a search with aggregations
17
Plugin Development
Plugin Development
Introduction
Creating a plugin
Creating an analyzer plugin
Creating a REST plugin
Creating a cluster action
Creating an ingest plugin
18
Big Data Integration
Big Data Integration
Introduction
Installing Apache Spark
Indexing data via Apache Spark
Indexing data with meta via Apache Spark
Reading data with Apache Spark
Reading data using SparkSQL
Indexing data with Apache Pig

Shrinking an index

The latest version of Elasticsearch provides a new way to optimize the index, via the shrink API it's possible to reduce the number of shards of an index.

This feature targets several common scenarios:

  • The wrong number of shards during the initial design sizing. Often sizing the shards without knowing the correct data/text distribution tends to oversize the number of shards

  • Reducing the number of shards to reduce memory and resource usage

  • Reducing the number of shards to speed up searching

Getting ready

You need an up-and-running Elasticsearch installation, as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.

To execute curl via the command line, you need to install curl for your operative system.

To correctly execute the following commands, use the index created in the Creating an index recipe.

How to do it...

The HTTP method used is POST. The URL format for optimizing one or more indices, is:

http://<server>/<source_index_name...

Previous Section
End of Section 13
Next Section