Book Image

Machine Learning with the Elastic Stack

By : Rich Collier, Bahaaldine Azarmi
Book Image

Machine Learning with the Elastic Stack

By: Rich Collier, Bahaaldine Azarmi

Overview of this book

Machine Learning with the Elastic Stack is a comprehensive overview of the embedded commercial features of anomaly detection and forecasting. The book starts with installing and setting up Elastic Stack. You will perform time series analysis on varied kinds of data, such as log files, network flows, application metrics, and financial data. As you progress through the chapters, you will deploy machine learning within the Elastic Stack for logging, security, and metrics. In the concluding chapters, you will see how machine learning jobs can be automatically distributed and managed across the Elasticsearch cluster and made resilient to failure. By the end of this book, you will understand the performance aspects of incorporating machine learning within the Elastic ecosystem and create anomaly detection jobs and view results from Kibana directly.

Related Content you might be interested in

Current Title:

Small book image
Machine Learning with the Elastic Stack
Rich Collier | Bahaaldine Azarmi
Jan, 2019 | 304 pages
Small book image
Learning Kibana 7
Bahaaldine Azarmi | Anurag Srivastava
Jul, 2019 | 280 pages
Small book image
Learning Kibana 5.0
Bahaaldine Azarmi
Feb, 2017 | 284 pages
Small book image
Mastering Kibana 6.x
Anurag Srivastava
Jul, 2018 | 376 pages
Table of Contents (12 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Machine Learning for IT
Machine Learning for IT
Overcoming the historical challenges
Theory of operation
Operationalization
Supporting indices
The orchestration
Summary
2
Installing the Elastic Stack with Machine Learning
Installing the Elastic Stack with Machine Learning
Installing the Elastic Stack
A guided tour of Elastic ML features
Summary
3
Event Change Detection
Event Change Detection
How to understand the normal rate of occurrence
Exploring count functions
Counting in population analysis
Detecting things that rarely occur
Counting message-based logs via categorization
Summary
4
IT Operational Analytics and Root Cause Analysis
IT Operational Analytics and Root Cause Analysis
Holistic application visibility
Data organization
Bringing it all together for root cause analysis
Summary
5
Security Analytics with Elastic Machine Learning
Security Analytics with Elastic Machine Learning
Security in the field
Threat hunting architecture
Investigation analytics
Summary
6
Alerting on ML Analysis
Alerting on ML Analysis
Results presentation
The results index
Alerts from the Machine Learning UI in Kibana
Creating ML alerts manually
Summary
7
Using Elastic ML Data in Kibana Dashboards
Using Elastic ML Data in Kibana Dashboards
Visualization options in Kibana
Preparing data for anomaly detection analysis
Building the visualizations
Summary
8
Using Elastic ML with Kibana Canvas
Using Elastic ML with Kibana Canvas
Introduction to Canvas
Building Elastic ML Canvas slides
Summary
9
Forecasting
Forecasting
Forecasting versus prophesying
Forecasting use cases
Forecasting – theory of operation
Single time series forecasting
Forecast results
Multiple time series forecasting
Summary
10
ML Tips and Tricks
ML Tips and Tricks
Job groups
Influencers in split versus non-split jobs
Using ML on scripted fields
Using one-sided ML functions to your advantage
Ignoring time periods
Don't over-engineer the use case
ML job throughput considerations
Top-down alerting by leveraging custom rules
Sizing ML deployments
Summary
11
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Sizing ML deployments

People often ask how they should appropriately size their cluster if they plan on using Elastic ML. Other than the obvious it depends answer, it is useful to have an empirical approach to the process. As seen on the Elastic blog Sizing for Machine Learning with Elasticsearch (https://www.elastic.co/blog/sizing-machine-learning-with-elasticsearch), there is a key recommendation: use dedicated nodes for ML so that you don't have ML jobs interfere with the other tasks of the data nodes of a cluster (indexing, searching, and so on). To scope how many dedicated nodes are necessary, follow this approach:

  • If there are no representative jobs created yet, use generic rules of thumb based on the overall cluster size from the blog. These rules of thumb are as follows:
    • Recommend 1 dedicated ML node (2 for HA) for cluster sizes < 10 data nodes
    • Definitely at...
Previous Section
End of Section 10
Next Section