Book Image

Machine Learning with the Elastic Stack

By : Rich Collier, Bahaaldine Azarmi
Book Image

Machine Learning with the Elastic Stack

By: Rich Collier, Bahaaldine Azarmi

Overview of this book

Machine Learning with the Elastic Stack is a comprehensive overview of the embedded commercial features of anomaly detection and forecasting. The book starts with installing and setting up Elastic Stack. You will perform time series analysis on varied kinds of data, such as log files, network flows, application metrics, and financial data. As you progress through the chapters, you will deploy machine learning within the Elastic Stack for logging, security, and metrics. In the concluding chapters, you will see how machine learning jobs can be automatically distributed and managed across the Elasticsearch cluster and made resilient to failure. By the end of this book, you will understand the performance aspects of incorporating machine learning within the Elastic ecosystem and create anomaly detection jobs and view results from Kibana directly.

Related Content you might be interested in

Current Title:

Small book image
Machine Learning with the Elastic Stack
Rich Collier | Bahaaldine Azarmi
Jan, 2019 | 304 pages
Small book image
Learning Kibana 7
Bahaaldine Azarmi | Anurag Srivastava
Jul, 2019 | 280 pages
Small book image
Learning Kibana 5.0
Bahaaldine Azarmi
Feb, 2017 | 284 pages
Small book image
Mastering Kibana 6.x
Anurag Srivastava
Jul, 2018 | 376 pages
Table of Contents (12 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Machine Learning for IT
Machine Learning for IT
Overcoming the historical challenges
Theory of operation
Operationalization
Supporting indices
The orchestration
Summary
2
Installing the Elastic Stack with Machine Learning
Installing the Elastic Stack with Machine Learning
Installing the Elastic Stack
A guided tour of Elastic ML features
Summary
3
Event Change Detection
Event Change Detection
How to understand the normal rate of occurrence
Exploring count functions
Counting in population analysis
Detecting things that rarely occur
Counting message-based logs via categorization
Summary
4
IT Operational Analytics and Root Cause Analysis
IT Operational Analytics and Root Cause Analysis
Holistic application visibility
Data organization
Bringing it all together for root cause analysis
Summary
5
Security Analytics with Elastic Machine Learning
Security Analytics with Elastic Machine Learning
Security in the field
Threat hunting architecture
Investigation analytics
Summary
6
Alerting on ML Analysis
Alerting on ML Analysis
Results presentation
The results index
Alerts from the Machine Learning UI in Kibana
Creating ML alerts manually
Summary
7
Using Elastic ML Data in Kibana Dashboards
Using Elastic ML Data in Kibana Dashboards
Visualization options in Kibana
Preparing data for anomaly detection analysis
Building the visualizations
Summary
8
Using Elastic ML with Kibana Canvas
Using Elastic ML with Kibana Canvas
Introduction to Canvas
Building Elastic ML Canvas slides
Summary
9
Forecasting
Forecasting
Forecasting versus prophesying
Forecasting use cases
Forecasting – theory of operation
Single time series forecasting
Forecast results
Multiple time series forecasting
Summary
10
ML Tips and Tricks
ML Tips and Tricks
Job groups
Influencers in split versus non-split jobs
Using ML on scripted fields
Using one-sided ML functions to your advantage
Ignoring time periods
Don't over-engineer the use case
ML job throughput considerations
Top-down alerting by leveraging custom rules
Sizing ML deployments
Summary
11
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

How to understand the normal rate of occurrence

Imagine that you're troubleshooting a problem by looking at a particular log file. You see a line in the log that looks like the following:

18/05/2017 15:16:00 DB Not Updated [Master] Table

Unless you have some intimate knowledge about the inner workings of the application that created this log, you may not know whether the message is important. Having the database be Not Updated possibly sounds like a negative situation. However, if you knew that the application routinely writes this message, day in and day out, several hundred times per hour, then you would naturally realize that this message is benign and should possibly be ignored, because clearly the application works fine every day despite this message being written to the log file.

The problem, obviously, is one of human interpretation. Inspection of the text of the...

Previous Section
End of Section 2
Next Section