Book Image

Practical Deep Learning at Scale with MLflow

By : Yong Liu
5 (1)
Book Image

Practical Deep Learning at Scale with MLflow

5 (1)
By: Yong Liu

Overview of this book

The book starts with an overview of the deep learning (DL) life cycle and the emerging Machine Learning Ops (MLOps) field, providing a clear picture of the four pillars of deep learning: data, model, code, and explainability and the role of MLflow in these areas. From there onward, it guides you step by step in understanding the concept of MLflow experiments and usage patterns, using MLflow as a unified framework to track DL data, code and pipelines, models, parameters, and metrics at scale. You’ll also tackle running DL pipelines in a distributed execution environment with reproducibility and provenance tracking, and tuning DL models through hyperparameter optimization (HPO) with Ray Tune, Optuna, and HyperBand. As you progress, you’ll learn how to build a multi-step DL inference pipeline with preprocessing and postprocessing steps, deploy a DL inference pipeline for production using Ray Serve and AWS SageMaker, and finally create a DL explanation as a service (EaaS) using the popular Shapley Additive Explanations (SHAP) toolbox. By the end of this book, you’ll have built the foundation and gained the hands-on experience you need to develop a DL pipeline solution from initial offline experimentation to final deployment and production, all within a reproducible and open source framework.

Related Content you might be interested in

Current Title:

Small book image
Practical Deep Learning at Scale with MLflow
Yong Liu
Jul, 2022 | 288 pages
Small book image
Machine Learning Engineering with MLflow.
Natu Lauchande
Aug, 2021 | 248 pages
Small book image
Practical Machine Learning on Databricks
Debu Sinha
Nov, 2023 | 244 pages
Small book image
Distributed Data Systems with Azure Databricks
Alan Bernardo Palacio
May, 2021 | 414 pages
Table of Contents (17 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Section 1 - Deep Learning Challenges and MLflow Prime
Section 1 - Deep Learning Challenges and MLflow Prime
Free Chapter
2
Chapter 1: Deep Learning Life Cycle and MLOps Challenges
Chapter 1: Deep Learning Life Cycle and MLOps Challenges
Technical requirements
Understanding the DL life cycle and MLOps challenges
Understanding DL data challenges
Understanding DL model challenges
Understanding DL code challenges
Understanding DL explainability challenges
Summary
Further reading
3
Chapter 2: Getting Started with MLflow for Deep Learning
Chapter 2: Getting Started with MLflow for Deep Learning
Technical requirements
Setting up MLflow
Implementing our first DL experiment with MLflow autologging
Exploring MLflow's components and usage patterns
Summary
Further reading
4
Section 2 –
Tracking a Deep Learning Pipeline at Scale
Section 2 –
Tracking a Deep Learning Pipeline at Scale
5
Chapter 3: Tracking Models, Parameters, and Metrics
Chapter 3: Tracking Models, Parameters, and Metrics
Technical requirements
Setting up a full-fledged local MLflow tracking server
Tracking model provenance
Tracking model metrics
Tracking model parameters
Summary
Further reading
6
Chapter 4: Tracking Code and Data Versioning
Chapter 4: Tracking Code and Data Versioning
Technical requirements
Tracking notebook and pipeline versioning
Tracking locally, privately built Python libraries
Tracking data versioning in Delta Lake
Summary
Further reading
7
Section 3 –
Running Deep Learning Pipelines at Scale
Section 3 –
Running Deep Learning Pipelines at Scale
8
Chapter 5: Running DL Pipelines in Different Environments
Chapter 5: Running DL Pipelines in Different Environments
Technical requirements
An overview of different execution scenarios and environments
Running locally with local code
Running remote code in GitHub locally
Running local code remotely in the cloud
Running remotely in the cloud with remote code in GitHub
Summary
Further reading
9
Chapter 6: Running Hyperparameter Tuning at Scale
Chapter 6: Running Hyperparameter Tuning at Scale
Technical requirements
Understanding automatic HPO for DL pipelines
Creating HPO-ready DL models with Ray Tune and MLflow
Running the first Ray Tune HPO experiment with MLflow
Running HPO with Ray Tune using Optuna and HyperBand
Summary
Further reading
10
Section 4 –
Deploying a Deep Learning Pipeline at Scale
Section 4 –
Deploying a Deep Learning Pipeline at Scale
11
Chapter 7: Multi-Step Deep Learning Inference Pipeline
Chapter 7: Multi-Step Deep Learning Inference Pipeline
Technical requirements
Understanding patterns of DL inference pipelines
Implementing a custom MLflow Python model
Implementing preprocessing and postprocessing steps in a DL inference pipeline
Implementing an inference pipeline as a new entry point in the main MLproject
Summary
Further reading
12
Chapter 8: Deploying a DL Inference Pipeline at Scale
Chapter 8: Deploying a DL Inference Pipeline at Scale
Technical requirements
Understanding different deployment tools and host environments
Deploying locally for batch and web service inference
Deploying using Ray Serve and MLflow deployment plugins
Deploying to AWS SageMaker – a complete end-to-end guide
Summary
Further reading
13
Section 5 – Deep Learning Model Explainability at Scale
Section 5 – Deep Learning Model Explainability at Scale
14
Chapter 9: Fundamentals of Deep Learning Explainability
Chapter 9: Fundamentals of Deep Learning Explainability
Technical requirements
Understanding the categories and audience of explainability
Exploring the SHAP Explainability toolbox
Exploring the Transformers Interpret toolbox
Summary
Further reading
15
Chapter 10: Implementing DL Explainability with MLflow
Chapter 10: Implementing DL Explainability with MLflow
Technical requirements
Understanding current MLflow explainability integration
Implementing a SHAP explanation using the MLflow artifact logging API
Implementing a SHAP explainer using the MLflow pyfunc API
Summary
Further reading
Why subscribe?
16
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Understanding DL code challenges

In this section, we will discuss DL code challenges. Let's look at how these code challenges are manifested in each of the stages described in Figure 1.3. In this section, and within the context of DL development, code refers to the source code that's written in certain programming languages such as Python for data processing and implementation, while a model refers to the model logic and architecture in its serialized format (for example, pickle, TorchScript, or ONNX):

  • Data collection/cleaning/annotation: While data is the central piece in this stage, the code that does the query, extraction/transformation/loading (ETL), and data cleaning and augmentation is of critical importance. We cannot decouple the development of the model from the data pipelines that provide the data feeds to the model. Therefore, data pipelines that implement ETL need to be treated as one of the integrated steps in both offline experimentation and online production. A common mistake is that we use different data ETL and cleaning pipelines in offline experimentation, and then implement different data ETL/cleaning pipelines in online production, which could cause different model behaviors. We need to version and serialize the data pipeline as part of the entire model pipeline. MLflow provides several ways to allow us to implement such multistep pipelines.
  • Model development: During offline experiments, in addition to different versions of data pipeline code, we might also have different versions of notebooks or use different versions of DL library code. The usage of notebooks is particularly unique in ML/DL life cycles. Tracking which model results are produced by which notebook/model pipeline/data pipeline needs to be done for each run. MLflow does that with automatic code version tracking and dependencies. In addition, code reproducibility in different running environments is unique to DL models, as DL models usually require hardware accelerators such as GPUs or TPUs. The flexibility of running either locally, or remotely, on a CPU or GPU environment is of great importance. MLflow provides a lightweight approach in which to organize the ML projects so that code can be written once and run everywhere.
  • Model deployment and serving in production: While the model is serving production traffic, any bugs will need to be traced back to both the model and code. Thus, tracking code provenance is critical. It is also critical to track all the dependency code library versions for a particular version of the model.
  • Model validation and A/B testing: Online experiments could use multiple versions of models using different data feeds. Debugging any experimentation will require not only knowing which model is used but also which code is used to produce that model.
  • Monitoring and feedback loops: This stage is similar to the previous stage in terms of code challenges, where we need to know whether model degradation is due to code bugs or model and data drifting. The monitoring pipeline needs to collect all the metrics for both data and model performance.

In summary, DL code challenges are especially unique because DL frameworks are still evolving (for example, TensorFlow, PyTorch, Keras, Hugging Face, and SparkNLP). MLflow provides a lightweight framework to overcome many common challenges and can interface with many DL frameworks seamlessly.

Previous Section
End of Section 6
Next Section