8. Handling Time-based Data | ElasticSearch Blueprints

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents
Feedback & Rating

ElasticSearch Blueprints

By : Mohan

3 (2)

ElasticSearch Blueprints

3 (2)

By: Mohan

Overview of this book

If you are a data enthusiast and would like to explore and specialize on search technologies based on Elasticsearch, this is the right book for you. A compelling case-to-case mapping of features and implementation of Elasticsearch to solve many real-world use cases makes this book the right choice to start and specialize on Elasticsearch.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Free Chapter

1. Google-like Web Search

1. Google-like Web Search

Deploying Elasticsearch

Communicating with the Elasticsearch server

Setting the analyzer

Using phrase query to search

Using the highlighting feature

Pagination

Summary

2. Building Your Own E-Commerce Solution

2. Building Your Own E-Commerce Solution

Data modeling in Elasticsearch

Choosing between a query and a filter

Searching your documents

Aggregating your results

Filter your results based on a date range

Implementing a prize range filter

Implementing a category filter

Implementation of filters in Elasticsearch

Searching with multiple conditions

Sorting results

Using the scroll API for consistent pagination

Autocomplete in Elasticsearch

Hotel suggester using autocomplete

Summary

3. Relevancy and Scoring

3. Relevancy and Scoring

How scoring works

The Ebola outbreak

Summary

4. Managing Relational Content

4. Managing Relational Content

The product-with-tags search problem

Nested types to the rescue

Limitations on a query on nested fields

Using a parent-child approach

Schema design to store questions and answers

Searching questions based on a criteria of answers

Searching answers based on a criteria of questions

The score of questions based on the score of each answer

Filtering questions with more than four answers

Summary

5. Analytics Using Elasticsearch

5. Analytics Using Elasticsearch

A flight ticket analytics scenario

Summary

6. Improving the Search Experience

6. Improving the Search Experience

News search

A case-insensitive search

Effective e-mail or URL link search inside text

Prioritizing a title match over content match

Terms aggregation giving weird results

Using a lowercased analyzer

Improving the search experience using stemming

A synonym-aware search

The holy box of search

Boolean operations

Words with similar sounds

Substring matching

Summary

7. Spicing Up a Search Using Geo

7. Spicing Up a Search Using Geo

Restaurant search

Data modeling for restaurants

The nearest hotel problem

The maximum distance covered

Inside the city limits

Distance values between the current point and each restaurant

Restaurant categorization based on distance

Aggregating restaurants based on their nearness

Summary

8. Handling Time-based Data

8. Handling Time-based Data

Overriding default mapping and settings in Elasticsearch

Index template creation

Searching for time-based data

Archiving time-based data

Closing older indices

Snapshot creation

Restoring a snapshot

Restoring multiple indices

The curator

Shard allocation using curator

Optimization

Summary

Index

Index

Customer Reviews

3 (2)

5 star

50%

4 star

0%

3 star

0%

2 star

0%

1 star

50%

Archiving time-based data

While dealing with time-based data, it is often noticed that the most useful data is that of the present. This makes the old data less relevant for our purposes. So as time progresses, the relevancy of past data falls very rapidly and the data we indexed exists without being used in the clusters. This situation is not very resource friendly, as there would be much unused data stored for no or less purpose.

We can visualize different levels of archiving, as follows:

Keep the hottest index in the machines that have good hardware (shard filtering).
Run the optimized API on indices where writing is done.
Close indices that are not required for instant search.
Take a snapshot and archive older indices.
Finally, remove indices that are no longer required.

Shard filtering

With time-based data, recent indices are more frequently used or are more relevant. In other words, at a given time, the data flowing would use some specific index based on the day, week, or month. This index...

Visually different images

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

ElasticSearch Blueprints

Search

Your notes and bookmarks