Book Image

Machine Learning with the Elastic Stack

By : Rich Collier, Bahaaldine Azarmi
Book Image

Machine Learning with the Elastic Stack

By: Rich Collier, Bahaaldine Azarmi

Overview of this book

Machine Learning with the Elastic Stack is a comprehensive overview of the embedded commercial features of anomaly detection and forecasting. The book starts with installing and setting up Elastic Stack. You will perform time series analysis on varied kinds of data, such as log files, network flows, application metrics, and financial data. As you progress through the chapters, you will deploy machine learning within the Elastic Stack for logging, security, and metrics. In the concluding chapters, you will see how machine learning jobs can be automatically distributed and managed across the Elasticsearch cluster and made resilient to failure. By the end of this book, you will understand the performance aspects of incorporating machine learning within the Elastic ecosystem and create anomaly detection jobs and view results from Kibana directly.
Table of Contents (12 chapters)

Counting message-based logs via categorization

If you have log entries that are message-based, but are machine-generated, then before they can be useful for anomaly detection, they first need to be organized into similar message types. This process is called categorization and Elastic ML can help with that process.

Types of messages that can be categorized by ML

We need to be a little rigorous in our definition of the kinds of message-based log lines that are being considered here. What we are not considering are log lines/events/documents that are completely free-form and likely the result of human creation (emails, tweets, comments, and so on). These kinds of messages are too arbitrary and variable in their construction...