Book Image

Elasticsearch 8.x Cookbook - Fifth Edition

By : Alberto Paro
Book Image

Elasticsearch 8.x Cookbook - Fifth Edition

By: Alberto Paro

Overview of this book

Elasticsearch is a Lucene-based distributed search engine at the heart of the Elastic Stack that allows you to index and search unstructured content with petabytes of data. With this updated fifth edition, you'll cover comprehensive recipes relating to what's new in Elasticsearch 8.x and see how to create and run complex queries and analytics. The recipes will guide you through performing index mapping, aggregation, working with queries, and scripting using Elasticsearch. You'll focus on numerous solutions and quick techniques for performing both common and uncommon tasks such as deploying Elasticsearch nodes, using the ingest module, working with X-Pack, and creating different visualizations. As you advance, you'll learn how to manage various clusters, restore data, and install Kibana to monitor a cluster and extend it using a variety of plugins. Furthermore, you'll understand how to integrate your Java, Scala, Python, and big data applications such as Apache Spark and Pig with Elasticsearch and create efficient data applications powered by enhanced functionalities and custom plugins. By the end of this Elasticsearch cookbook, you'll have gained in-depth knowledge of implementing the Elasticsearch architecture and be able to manage, search, and store data efficiently and effectively using Elasticsearch.
Table of Contents (20 chapters)

Mapping the Rank Feature and Feature Vector fields

It's common to want to score a document dynamically, depending on the context. For example, if you need to score more documents that are inside a category, the classic scenario is to boost (increase low-scored) documents that are based on a value, such as page rank, hits, or categories.

Elasticsearch provides two new ways to boost your scores based on values. One is the Rank Feature field, while the other is its extension, which is to use a vector of values.

Getting ready

You will need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe of Chapter 1Getting Started.

To execute the commands in this recipe, you can use any HTTP client, such as curl (, Postman (, or similar. I suggest using the Kibana console, which provides code completion and better character escaping for Elasticsearch.

How to do it…

We want to use the rank_feature type to implement a common PageRank scenario where documents are scored based on the same characteristics. To achieve this, follow these steps:

  1. To be able to score based on a pagerank value and an inverse url length, we can use the following mapping:
    PUT test-rank
    {  "mappings": {
        "properties": {
          "pagerank": { "type": "rank_feature" },
          "url_length": {
            "type": "rank_feature",
            "positive_score_impact": false
    } } } }
  2. Now, we can store a document, as shown here:
    PUT test-rank/_doc/1
    { "pagerank": 5, "url_length": 20 }
  3. Now, we can execute a feature query on the pagerank value to return our record with a similar query, like so:
    GET test-rank/_search
    { "query": { "rank_feature": { "field":"pagerank" }}} 

    Important Note

    To query the special rank/rank_features types, we need to use the special rank_feature query type, which is only used for this special case.

The evolution of the previous feature's functionality is to define a vector of values using the rank_features type; usually, it can be used to score by topics, categories, or similar discerning facets. We can implement this functionality by following these steps:

  1. First, we must define the mapping for the categories field:
    PUT test-ranks
    { "mappings": {
        "properties": {
          "categories": { "type": "rank_features"  } } } }
  2. Now, we can store some documents in the index by using the following commands:
    PUT test-ranks/_doc/1
    { "categories": { "sport": 14.2, "economic": 24.3 } }
    PUT test-ranks/_doc/2
    { "categories": { "sport": 19.2, "economic": 23.1 } }
  3. Now, we can search based on the saved feature values, as shown here:
    GET test-ranks/_search
    { "query": { "feature": { "field": ""   } } }

How it works…

rank_feature and rank_features are special type fields that are used for storing values and are mainly used to score the results.

Important Note

The values that are stored in these fields can only be queried using the feature query. This cannot be used in standard queries and aggregations.

The value numbers in rank_feature and rank_features can only be single positive values (multi-values are not allowed).

In the case of rank_features, the values must be a hash, composed of a string and a positive numeric value.

There is a flag that changes the behavior of scoring – positive_score_impact. This value is true by default, but if you want the value of the feature to decrease the score, you can set it to false. In the pagerank example, the length of url reduces the score of the document because the longer url is, the less relevant it becomes.