Chapter 12: Using the Ingest Module | Elasticsearch 8.x Cookbook

Book Overview & Buying
Table Of Contents

Elasticsearch 8.x Cookbook - Fifth Edition

By : Alberto Paro

4 (6)

Buy this Book

Elasticsearch 8.x Cookbook

4 (6)

By: Alberto Paro

Buy this Book

Overview of this book

Elasticsearch is a Lucene-based distributed search engine at the heart of the Elastic Stack that allows you to index and search unstructured content with petabytes of data. With this updated fifth edition, you'll cover comprehensive recipes relating to what's new in Elasticsearch 8.x and see how to create and run complex queries and analytics. The recipes will guide you through performing index mapping, aggregation, working with queries, and scripting using Elasticsearch. You'll focus on numerous solutions and quick techniques for performing both common and uncommon tasks such as deploying Elasticsearch nodes, using the ingest module, working with X-Pack, and creating different visualizations. As you advance, you'll learn how to manage various clusters, restore data, and install Kibana to monitor a cluster and extend it using a variety of plugins. Furthermore, you'll understand how to integrate your Java, Scala, Python, and big data applications such as Apache Spark and Pig with Elasticsearch and create efficient data applications powered by enhanced functionalities and custom plugins. By the end of this Elasticsearch cookbook, you'll have gained in-depth knowledge of implementing the Elasticsearch architecture and be able to manage, search, and store data efficiently and effectively using Elasticsearch.

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the color images

Conventions used

Sections

Get in touch

Share Your Thoughts

Chapter 1: Getting Started

Technical requirements

Downloading and installing Elasticsearch

Setting up networking

Setting up a node

Setting up Linux systems

Setting up different node roles

Setting up a coordinating-only node

Setting up an ingestion node

Installing plugins in Elasticsearch

Removing a plugin

Changing logging settings

Setting up a node via Docker

Deploying on Elastic Cloud Enterprise

Free Chapter

Chapter 2: Managing Mappings

Technical requirements

Using explicit mapping creation

Mapping base types

Mapping arrays

Mapping an object

Mapping a document

Using dynamic templates in document mapping

Managing nested objects

Managing a child document with a join field

Adding a field with multiple mappings

Mapping a GeoPoint field

Mapping a GeoShape field

Mapping an IP field

Mapping an Alias field

Mapping a Percolator field

Mapping the Rank Feature and Feature Vector fields

Mapping the Search as you type field

Using the Range Fields type

Using the Flattened field type

Using the Point and Shape field types

Using the Dense Vector field type

Using the Histogram field type

Adding metadata to a mapping

Specifying different analyzers

Using index components and templates

Chapter 3: Basic Operations

Technical requirements

Creating an index

Deleting an index

Opening or closing an index

Putting a mapping in an index

Getting a mapping

Reindexing an index

Refreshing an index

Flushing an index

Using ForceMerge on an index

Shrinking an index

Checking whether an index exists

Managing index settings

Using index aliases

Managing dangling indices

Resolving index names

Rolling over an index

Indexing a document

Getting a document

Deleting a document

Updating a document

Speeding up atomic operations (bulk operations)

Speeding up GET operations (multi-GET)

Chapter 4: Exploring Search Capabilities

Technical requirements

Executing a search

Sorting results

Highlighting results

Executing a scrolling query

Using the search_after functionality

Returning inner hits in results

Suggesting a correct query

Counting matched results

Explaining a query

Query profiling

Deleting by query

Updating by query

Matching all of the documents

Using a Boolean query

Using the search template

Chapter 5: Text and Numeric Queries

Technical requirements

Using a term query

Using a terms query

Using a terms set query

Using a prefix query

Using a wildcard query

Using a regexp query

Using span queries

Using a match query

Using a query string query

Using a simple query string query

Using the range query

Using an IDs query

Using the function score query

Using the exists query

Using a pinned query (XPACK)

Chapter 6: Relationships and Geo Queries

Technical requirements

Using the has_child query

Using the has_parent query

Using the nested query

Using the geo_bounding_box query

Using the geo_shape query

Using the geo_distance query

Chapter 7: Aggregations

Executing an aggregation

Executing a stats aggregation

Executing a terms aggregation

Executing a significant terms aggregation

Executing a range aggregation

Executing a histogram aggregation

Executing a date histogram aggregation

Executing a filter aggregation

Executing a filters aggregation

Executing a global aggregation

Executing a geo distance aggregation

Executing a children aggregation

Executing a nested aggregation

Executing a top hit aggregation

Executing a matrix stats aggregation

Executing a geo bounds aggregation

Executing a geo centroid aggregation

Executing a geotile grid aggregation

Executing a sampler aggregation

Executing a pipeline aggregation

Chapter 8: Scripting in Elasticsearch

Painless scripting

Installing additional scripting languages

Managing scripts

Sorting data using scripts

Computing return fields with scripting

Filtering a search using scripting

Using scripting in aggregations

Updating a document using scripts

Reindexing with a script

Scripting in ingest processors

Chapter 9: Managing Clusters

Controlling the cluster health using the health API

Controlling the cluster state using the API

Getting cluster node information using the API

Getting node statistics using the API

Using the task management API

Using the hot threads API

Managing the shard allocation

Monitoring segments with the segment API

Cleaning the cache

Chapter 10: Backups and Restoring Data

Managing repositories

Executing a snapshot

Restoring a snapshot

Setting up an NFS share for backups

Reindexing from a remote cluster

Chapter 11: User Interfaces

Installing Kibana

Managing Kibana Discover

Visualizing data with Kibana

Using Kibana Dev Tools

Chapter 12: Using the Ingest Module

Pipeline definition

Inserting an ingest pipeline

Getting an ingest pipeline

Deleting an ingest pipeline

Simulating an ingest pipeline

Built-in processors

The grok processor

Using the ingest attachment plugin

Using the ingest GeoIP processor

Using the enrichment processor

Chapter 13: Java Integration

Creating a standard Java HTTP client

Creating a low-level Elasticsearch client

Using the Elasticsearch official Java client

Managing indices

Managing mappings

Managing documents

Managing bulk actions

Building a query

Executing a standard search

Executing a search with aggregations

Executing a scroll search

Integrating with DeepLearning4j

Chapter 14: Scala Integration

Creating a client in Scala

Managing indices

Managing mappings

Managing documents

Executing a standard search

Executing a search with aggregations

Integrating with DeepLearning.scala

Chapter 15: Python Integration

Creating a client

Managing indices

Managing mappings

Managing documents

Executing a standard search

Executing a search with aggregations

Integrating with NumPy and scikit-learn

Using AsyncElasticsearch

Using Elasticsearch with FastAPI

Chapter 16: Plugin Development

Creating a plugin

Creating an analyzer plugin

Creating a REST plugin

Creating a cluster action

Creating an ingest plugin

Chapter 17: Big Data Integration

Installing Apache Spark

Indexing data using Apache Spark

Indexing data with meta using Apache Spark

Reading data with Apache Spark

Reading data using Spark SQL

Indexing data with Apache Pig

Using Elasticsearch with Alpakka

Using Elasticsearch with MongoDB

Chapter 18: X-Pack

ILM – managing the index life cycle

ILM – automating rollover

Using the SQL Rest API

Using SQL via JDBC

Using X-Pack Security

Using alerting to monitor data events

Why subscribe?

Other Books You May Enjoy

Packt is searching for authors like you

Share Your Thoughts

Elasticsearch 8.x Cookbook - Fifth Edition

By : Alberto Paro

Elasticsearch 8.x Cookbook

By: Alberto Paro

Overview of this book

Getting an ingest pipeline

Getting ready

How to do it...

Elasticsearch 8.x Cookbook - Fifth Edition

By : Alberto Paro

Elasticsearch 8.x Cookbook

By: Alberto Paro

Overview of this book

Getting an ingest pipeline

Getting ready

How to do it...

Confirmation

Buy this book with your credits?

Submit Your Feedback

Create a Free Account To Continue Reading

Sign in to activate your 7-day free access