Book Image

Machine Learning for Streaming Data with Python

By : Joos Korstanje
Book Image

Machine Learning for Streaming Data with Python

By: Joos Korstanje

Overview of this book

Streaming data is the new top technology to watch out for in the field of data science and machine learning. As business needs become more demanding, many use cases require real-time analysis as well as real-time machine learning. This book will help you to get up to speed with data analytics for streaming data and focus strongly on adapting machine learning and other analytics to the case of streaming data. You will first learn about the architecture for streaming and real-time machine learning. Next, you will look at the state-of-the-art frameworks for streaming data like River. Later chapters will focus on various industrial use cases for streaming data like Online Anomaly Detection and others. As you progress, you will discover various challenges and learn how to mitigate them. In addition to this, you will learn best practices that will help you use streaming data to generate real-time insights. By the end of this book, you will have gained the confidence you need to stream data in your machine learning models.

Related Content you might be interested in

Current Title:

Small book image
Machine Learning for Streaming Data with Python
Joos Korstanje
Jul, 2022 | 258 pages
Small book image
Machine Learning for Time-Series with Python
Ben Auffarth
Oct, 2021 | 370 pages
Table of Contents (17 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Part 1: Introduction and Core Concepts of Streaming Data
Part 1: Introduction and Core Concepts of Streaming Data
Free Chapter
2
Chapter 1: An Introduction to Streaming Data
Chapter 1: An Introduction to Streaming Data
Technical requirements
A short history of data science
Working with streaming data
Real-time data formats and importing an example dataset in Python
Summary
Further reading
3
Chapter 2: Architectures for Streaming and Real-Time Machine Learning
Chapter 2: Architectures for Streaming and Real-Time Machine Learning
Technical requirements
Defining your analytics as a function
Understanding microservices architecture
Communicating between services through APIs
Demystifying the HTTP protocol
Building a simple API on AWS
Big data tools for real time streaming
Summary
Further reading
4
Chapter 3: Data Analysis on Streaming Data
Chapter 3: Data Analysis on Streaming Data
Technical requirements
Descriptive statistics on streaming data
Introduction to sampling theory
Overview of the main descriptive statistics
Real-time visualizations
Building basic alerting systems
Summary
Further reading
5
Part 2: Exploring Use Cases for Data Streaming
Part 2: Exploring Use Cases for Data Streaming
6
Chapter 4: Online Learning with River
Chapter 4: Online Learning with River
Technical requirements
What is online machine learning?
Using River for online learning
Summary
Further reading
7
Chapter 5: Online Anomaly Detection
Chapter 5: Online Anomaly Detection
Technical requirements
Defining anomaly detection
Exploring use cases of anomaly detection
Comparing anomaly detection and imbalanced classification
Algorithms for detecting anomalies in River
Going further with anomaly detection
Summary
Further reading
8
Chapter 6: Online Classification
Chapter 6: Online Classification
Technical requirements
Defining classification
Identifying use cases of classification
Overview of classification algorithms in River
Summary
Further reading
9
Chapter 7: Online Regression
Chapter 7: Online Regression
Technical requirements
Defining regression
Use cases of regression
Overview of regression algorithms in River
Summary
Further reading
10
Chapter 8: Reinforcement Learning
Chapter 8: Reinforcement Learning
Technical requirements
Defining reinforcement learning
The main steps of a reinforcement learning model
Exploring Q-learning
Deep Q-learning
Using reinforcement learning for streaming data
Use cases of reinforcement learning
Implementing reinforcement learning in Python
Summary
Further reading
11
Part 3: Advanced Concepts and Best Practices around Streaming Data
Part 3: Advanced Concepts and Best Practices around Streaming Data
12
Chapter 9: Drift and Drift Detection
Chapter 9: Drift and Drift Detection
Technical requirements
Defining drift
Introducing model explicability
Measuring drift
Measuring drift in Python
Counteracting drift
Summary
Further reading
13
Chapter 10: Feature Transformation and Scaling
Chapter 10: Feature Transformation and Scaling
Technical requirements
Challenges of data preparation with streaming data
Scaling data for streaming
Transforming features in a streaming context
Summary
Further reading
14
Chapter 11: Catastrophic Forgetting
Chapter 11: Catastrophic Forgetting
Technical requirements
Introducing catastrophic forgetting
Catastrophic forgetting in online models
Detecting catastrophic forgetting
Model explicability versus catastrophic forgetting
Summary
Further reading
15
Chapter 12: Conclusion and Best Practices
Chapter 12: Conclusion and Best Practices
Going further
Summary
Why subscribe?
16
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Defining drift

It is a well-known and commonly observed problem that models tend to start performing worse with time. Whether your metric is accuracy, R2 score, F1 score, or anything else, you will see a slow but steady decrease in performance over time if you put models into production and do not update them.

Depending on your use case, this decrease may become problematic quickly or slowly. Some use cases need to have continuous, near-perfect predictions. In some use cases— for example, for specialized ML in which the models have a direct impact on life—you would be strongly shocked if you observed a 1 percent decrease. In other use cases, ML is used more as automation than as prediction, and the business partners may not even notice a 5 percent decrease.

Whether it is going to impact you is not the question here. What is sure, is that in general, you will see your models decreasing. The goal for this chapter is to make sure to find out why model performance is...

Previous Section
End of Section 3
Next Section