Book Image

Advanced Elasticsearch 7.0

By : Wai Tak Wong
Book Image

Advanced Elasticsearch 7.0

By: Wai Tak Wong

Overview of this book

Building enterprise-grade distributed applications and executing systematic search operations call for a strong understanding of Elasticsearch and expertise in using its core APIs and latest features. This book will help you master the advanced functionalities of Elasticsearch and understand how you can develop a sophisticated, real-time search engine confidently. In addition to this, you'll also learn to run machine learning jobs in Elasticsearch to speed up routine tasks. You'll get started by learning to use Elasticsearch features on Hadoop and Spark and make search results faster, thereby improving the speed of query results and enhancing the customer experience. You'll then get up to speed with performing analytics by building a metrics pipeline, defining queries, and using Kibana for intuitive visualizations that help provide decision-makers with better insights. The book will later guide you through using Logstash with examples to collect, parse, and enrich logs before indexing them in Elasticsearch. By the end of this book, you will have comprehensive knowledge of advanced topics such as Apache Spark support, machine learning using Elasticsearch and scikit-learn, and real-time analytics, along with the expertise you need to increase business productivity, perform analytics, and get the very best out of Elasticsearch.

Related Content you might be interested in

Current Title:

Small book image
Advanced Elasticsearch 7.0
Wai Tak Wong
Aug, 2019 | 560 pages
Small book image
Elasticsearch 7 Quick Start Guide
Douglas Miller | Anurag Srivastava
Oct, 2019 | 186 pages
Small book image
Learning Elasticsearch
Abhishek Andhavarapu
Jun, 2017 | 404 pages
Small book image
Elasticsearch 8.x Cookbook
Alberto Maria Angelo Paro
May, 2022 | 750 pages
Table of Contents (25 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Section 1: Fundamentals and Core APIs
Section 1: Fundamentals and Core APIs
2
Overview of Elasticsearch 7
Overview of Elasticsearch 7
Preparing your environment
Running Elasticsearch
Talking to Elasticsearch
Elasticsearch architectural overview
Key concepts
API conventions
New features
Breaking changes
Migration between versions
Summary
3
Index APIs
Index APIs
Index management APIs
Index settings
Index aliases
Monitoring indices
Index persistence
Advanced index management APIs
Summary
4
Document APIs
Document APIs
The Elasticsearch document life cycle
Single document management APIs
Multi-document management APIs
Migration from a multiple mapping types index
Summary
5
Mapping APIs
Mapping APIs
Dynamic mapping
Meta fields in mapping
Field datatypes
Mapping parameters
Refreshing mapping changes for static mapping
Typeless APIs working with old custom index types
Summary
6
Anatomy of an Analyzer
Anatomy of an Analyzer
An analyzer's components
Character filters
Tokenizers
Token filters
Built-in analyzers
Custom analyzers
Normalizers
Summary
7
Search APIs
Search APIs
Indexing sample documents
Search APIs
Query DSL
The multi-search API
Other search-related APIs
Summary
8
Section 2: Data Modeling, Aggregations Framework, Pipeline, and Data Analytics
Section 2: Data Modeling, Aggregations Framework, Pipeline, and Data Analytics
9
Modeling Your Data in the Real World
Modeling Your Data in the Real World
The Investor Exchange Cloud
Modeling data and the approaches
Practical considerations
Summary
10
Aggregation Frameworks
Aggregation Frameworks
ETF historical data preparation
Aggregation query syntax
Matrix aggregations
Metrics aggregations
Bucket aggregations
Pipeline aggregations
Post filter on aggregations
Summary
11
Preprocessing Documents in Ingest Pipelines
Preprocessing Documents in Ingest Pipelines
Ingest APIs
Accessing data in pipelines
Processors
Conditional execution in pipelines
Handling failures in pipelines
Summary
12
Using Elasticsearch for Exploratory Data Analysis
Using Elasticsearch for Exploratory Data Analysis
Business analytics
Operational data analytics
Sentiment analysis
Summary
13
Section 3: Programming with the Elasticsearch Client
Section 3: Programming with the Elasticsearch Client
14
Elasticsearch from Java Programming
Elasticsearch from Java Programming
Overview of Elasticsearch Java REST client
The Java low-level REST client
The Java high-level REST client
Spring Data Elasticsearch
Summary
15
Elasticsearch from Python Programming
Elasticsearch from Python Programming
Overview of the Elasticsearch Python client
The Python low-level Elasticsearch client
The Python high-level Elasticsearch library
Summary
16
Section 4: Elastic Stack
Section 4: Elastic Stack
17
Using Kibana, Logstash, and Beats
Using Kibana, Logstash, and Beats
Overview of the Elastic Stack
Running Elasticsearch in a Docker container
Running Kibana in a Docker container
Running Logstash in a Docker container
Running Beats in a Docker container
Summary
18
Working with Elasticsearch SQL
Working with Elasticsearch SQL
Overview
Getting started
Elasticsearch SQL language
Elasticsearch SQL REST API
Elasticsearch SQL JDBC
Summary
19
Working with Elasticsearch Analysis Plugins
Working with Elasticsearch Analysis Plugins
What are Elasticsearch plugins?
Working with the ICU Analysis plugin
Working with the Smart Chinese Analysis plugin
Working with the IK Analysis plugin
Summary
20
Section 5: Advanced Features
Section 5: Advanced Features
21
Machine Learning with Elasticsearch
Machine Learning with Elasticsearch
Machine learning with Elastic Stack
Machine learning using Elasticsearch and scikit-learn
Summary
22
Spark and Elasticsearch for Real-Time Analytics
Spark and Elasticsearch for Real-Time Analytics
Overview of ES-Hadoop
Apache Spark support
Real-time analytics using Elasticsearch and Apache Spark
Summary
23
Building Analytics RESTful Services
Building Analytics RESTful Services
Building a RESTful web service with Spring Boot
Integration with the Bollinger Band
Building a Java Spark ML module for k-means anomaly detection
Testing Analytics RESTful services
Working with Kibana to visualize the analytics results
Summary
24
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Overview of ES-Hadoop

As we mentioned, the ES-Hadoop feature contains two major areas: distributed computing and distributed storage. The main goal of ES-Hadoop is to seamlessly connect Elasticsearch and Hadoop so that they can benefit each other with distributed computing, distributed storage, searching, analytics, visualization, and more. We can import Hadoop Distributed File System (HDFS) data to Elasticsearch for search and analysis, and export the Elastisearch data to HDFS for snapshot and restore. ES-Hadoop fully supports the Spark framework, including Spark, Hive, Pig, Storm, Cascading, and sure, the standard MapReduce. Let's take a look at the data flow between Elasticsearch, ES-Hadoop, and components in the Hadoop ecosystem, as shown in the following screenshot:

In short, we can think of ES-Hadoop as a data bridge between Elasticsearch and the Hadoop big data ecosystem...

Previous Section
End of Section 2
Next Section