Book Image

Machine Learning with the Elastic Stack

By : Rich Collier, Bahaaldine Azarmi
Book Image

Machine Learning with the Elastic Stack

By: Rich Collier, Bahaaldine Azarmi

Overview of this book

Machine Learning with the Elastic Stack is a comprehensive overview of the embedded commercial features of anomaly detection and forecasting. The book starts with installing and setting up Elastic Stack. You will perform time series analysis on varied kinds of data, such as log files, network flows, application metrics, and financial data. As you progress through the chapters, you will deploy machine learning within the Elastic Stack for logging, security, and metrics. In the concluding chapters, you will see how machine learning jobs can be automatically distributed and managed across the Elasticsearch cluster and made resilient to failure. By the end of this book, you will understand the performance aspects of incorporating machine learning within the Elastic ecosystem and create anomaly detection jobs and view results from Kibana directly.
Table of Contents (12 chapters)

Using ML on scripted fields

In some cases, it might be necessary to analyze the relationship of fields within documents. Elasticsearch gives us the ability to create scripted fields (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-script-fields.html) that allow us to programmatically combine individual fields into new fields; ML can then leverage that!

For example, let's say you have daily documents that summarize your product sales and that your documents have two fields, Amount and Count:

{
...
    "Count": 160,
    "Amount": 7200
...
}

We could easily define a use case in which we were interested in the per item cost (Amount/Count). To do so, we would define the ML job to focus on the new field (perhaps we'll call it per_item_cost and use the mean function on it):

PUT _xpack/ml/anomaly_detectors/my_job {
    &quot...