Book Image

Elasticsearch 7.0 Cookbook - Fourth Edition

By : Alberto Paro

Book Image

Elasticsearch 7.0 Cookbook - Fourth Edition

By: Alberto Paro

Overview of this book

Elasticsearch is a Lucene-based distributed search server that allows users to index and search unstructured content with petabytes of data. With this book, you'll be guided through comprehensive recipes on what's new in Elasticsearch 7, and see how to create and run complex queries and analytics. Packed with recipes on performing index mapping, aggregation, and scripting using Elasticsearch, this fourth edition of Elasticsearch Cookbook will get you acquainted with numerous solutions and quick techniques for performing both every day and uncommon tasks such as deploying Elasticsearch nodes, integrating other tools to Elasticsearch, and creating different visualizations. You will install Kibana to monitor a cluster and also extend it using a variety of plugins. Finally, you will integrate your Java, Scala, Python, and big data applications such as Apache Spark and Pig with Elasticsearch, and create efficient data applications powered by enhanced functionalities and custom plugins. By the end of this book, you will have gained in-depth knowledge of implementing Elasticsearch architecture, and you'll be able to manage, search, and store data efficiently and effectively using Elasticsearch.

Title Page

Copyright and Credits

Copyright and Credits

Elasticsearch 7.0 Cookbook Fourth Edition

About Packt

Contributors

About the author

About the reviewer

Packt is searching for authors like you

Preface

Who this book is for

What this book covers

To get the most out of this book

Free Chapter

Getting Started

Getting Started

Technical requirements

Downloading and installing Elasticsearch

Setting up networking

Setting up a node

Setting up Linux systems

Setting up different node types

Setting up a coordinator node

Setting up an ingestion node

Installing plugins in Elasticsearch

Removing a plugin

Changing logging settings

Setting up a node via Docker

Deploying on Elasticsearch Cloud Enterprise

Managing Mapping

Managing Mapping

Using explicit mapping creation

Mapping base types

Mapping an object

Mapping a document

Using dynamic templates in document mapping

Managing nested objects

Managing a child document with a join field

Adding a field with multiple mappings

Mapping a GeoPoint field

Mapping a GeoShape field

Mapping an IP field

Mapping an alias field

Mapping a Percolator field

Mapping feature and feature vector fields

Adding metadata to a mapping

Specifying different analyzers

Mapping a completion field

Basic Operations

Basic Operations

Creating an index

Deleting an index

Opening or closing an index

Putting a mapping in an index

Getting a mapping

Reindexing an index

Refreshing an index

Flushing an index

ForceMerge an index

Shrinking an index

Checking if an index exists

Managing index settings

Using index aliases

Rolling over an index

Indexing a document

Getting a document

Deleting a document

Updating a document

Speeding up atomic operations (bulk operations)

Speeding up GET operations (multi GET)

Exploring Search Capabilities

Exploring Search Capabilities

Technical requirements

Executing a search

Sorting results

Highlighting results

Executing a scrolling query

Using the search_after functionality

Returning inner hits in results

Suggesting a correct query

Counting matched results

Explaining a query

Query profiling

Deleting by query

Updating by query

Matching all the documents

Using a Boolean query

Using the search template

Text and Numeric Queries

Text and Numeric Queries

Using a term query

Using a terms query

Using a prefix query

Using a wildcard query

Using a regexp query

Using span queries

Using a match query

Using a query string query

Using a simple query string query

Using the range query

The common terms query

Using an IDs query

Using the function score query

Using the exists query

Relationship and Geo Queries

Relationship and Geo Queries

Using the has_child query

Using the has_parent query

Using nested queries

Using the geo_bounding_box query

Using the geo_polygon query

Using the geo_distance query

Aggregations

Executing an aggregation

Executing stats aggregations

Executing terms aggregation

Executing significant terms aggregation

Executing range aggregations

Executing histogram aggregations

Executing date histogram aggregations

Executing filter aggregations

Executing filters aggregations

Executing global aggregations

Executing geo distance aggregations

Executing children aggregations

Executing nested aggregations

Executing top hit aggregations

Executing a matrix stats aggregation

Executing geo bounds aggregations

Executing geo centroid aggregations

Executing pipeline aggregations

Scripting in Elasticsearch

Scripting in Elasticsearch

Painless scripting

Installing additional script plugins

Managing scripts

Sorting data using scripts

Computing return fields with scripting

Filtering a search using scripting

Using scripting in aggregations

Updating a document using scripts

Reindexing with a script

Managing Clusters

Managing Clusters

Controlling the cluster health using an API

Controlling the cluster state using an API

Getting cluster node information using an API

Getting node statistics via the API

Using the task management API

Using the hot threads API

Managing the shard allocation

Monitoring segments with the segment API

Cleaning the cache

Backups and Restoring Data

Backups and Restoring Data

Managing repositories

Executing a snapshot

Restoring a snapshot

Setting up an NFS share for backups

Reindexing from a remote cluster

User Interfaces

User Interfaces

Installing and using Cerebro

Installing and using Elasticsearch HQ

Installing Kibana

Managing Kibana discovery

Visualizing data with Kibana

Using Kibana Dev tools

Using the Ingest Module

Using the Ingest Module

Pipeline definition

Inserting an ingest pipeline

Getting an ingest pipeline

Deleting an ingest pipeline

Simulating an ingest pipeline

Built-in processors

Using the ingest attachment plugin

Using the ingest GeoIP plugin

Java Integration

Java Integration

Creating a standard Java HTTP client

Creating an HTTP Elasticsearch client

Creating a high-level REST client

Managing indices

Managing mappings

Managing documents

Managing bulk actions

Building a query

Executing a standard search

Executing a search with aggregations

Executing a scroll search

Integrating with DeepLearning4j

Scala Integration

Scala Integration

Creating a client in Scala

Managing indices

Managing mappings

Managing documents

Executing a standard search

Executing a search with aggregations

Integrating with DeepLearning.scala

Python Integration

Python Integration

Creating a client

Managing indices

Managing mappings include the mapping

Managing documents

Executing a standard search

Executing a search with aggregations

Integrating with NumPy and scikit-learn

Plugin Development

Plugin Development

Creating a plugin

Creating an analyzer plugin

Creating a REST plugin

Creating a cluster action

Creating an ingest plugin

Big Data Integration

Big Data Integration

Installing Apache Spark

Indexing data using Apache Spark

Indexing data with meta using Apache Spark

Reading data with Apache Spark

Reading data using Spark SQL

Indexing data with Apache Pig

Using Elasticsearch with Alpakka

Using Elasticsearch with MongoDB

Another Book You May Enjoy

Another Book You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Setting up a coordinator node

The master nodes that we have seen previously are the most important for cluster stability. To prevent the queries and aggregations from creating instability in your cluster, coordinator (or client/proxy) nodes can be used to provide safe communication with the cluster.

Getting ready

You need a working Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in this chapter, and a simple text editor to change configuration files.

How to do it…

For the advance setup of a cluster, there are some parameters that must be configured to define different node types.

These parameters are in the config/elasticsearch.yml, file and they can be setup a coordinator node with the following steps:

Set up the node so that it's not a master, as follows:

node.master: false

Set up the node to not contain data, as follows:

node.data: false

How it works…

The coordinator node is a special node that works as a proxy/pass thought for the cluster. Its main advantages are as follows:

It can easily be killed or removed from the cluster without causing any problems. It's not a master, so it doesn't participate in cluster functionalities and it doesn't contain data, so there are no data relocations/replications due to its failure.
It prevents the instability of the cluster due to a developers' /users bad queries. Sometimes, a user executes aggregations that are too large (that is, date histograms with a range of some years and intervals of 10 seconds). Here, the Elasticsearch node could crash. (In its newest version, Elasticsearch has a structure called circuit breaker to prevent similar issues, but there are always borderline cases that can bring instability using scripting, for example. The coordinator node is not a master and its overload doesn't cause any problems for cluster stability.
If the coordinator or client node is embedded in the application, there are less round trips for the data, speeding up the application.
You can add them to balance the search and aggregation throughput without generating changes and data relocation in the cluster.