Book Image

Machine Learning with the Elastic Stack

By : Rich Collier, Bahaaldine Azarmi
Book Image

Machine Learning with the Elastic Stack

By: Rich Collier, Bahaaldine Azarmi

Overview of this book

Machine Learning with the Elastic Stack is a comprehensive overview of the embedded commercial features of anomaly detection and forecasting. The book starts with installing and setting up Elastic Stack. You will perform time series analysis on varied kinds of data, such as log files, network flows, application metrics, and financial data. As you progress through the chapters, you will deploy machine learning within the Elastic Stack for logging, security, and metrics. In the concluding chapters, you will see how machine learning jobs can be automatically distributed and managed across the Elasticsearch cluster and made resilient to failure. By the end of this book, you will understand the performance aspects of incorporating machine learning within the Elastic ecosystem and create anomaly detection jobs and view results from Kibana directly.
Table of Contents (12 chapters)

Top-down alerting by leveraging custom rules

In Chapter 4, IT Operational Analytics and Root Cause Analysis, we asked "what percentage of the data that you collect is being paid attention to?" Often, a realistic answer is likely <10% and maybe even <1%. The reason why this is the case is that the traditional approach to making data proactive is to start from scratch and then build up thresholds or rules-based alerts over time. This can be a daunting and/or tedious task that requires upfront knowledge (or at least a guess) as to what the expected behavior of each time series should be. Then, once the alerts have been configured, there can be an extended tuning process that balances alert sensitivity with annoying false positives. Additionally, there could also be metrics whose unusual behaviors could never be caught with a static threshold.

Combine this challenge...