Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Practical Deep Learning at Scale with MLflow
  • Table Of Contents Toc
Practical Deep Learning at Scale with MLflow

Practical Deep Learning at Scale with MLflow

By : Yong Liu
4.5 (11)
close
close
Practical Deep Learning at Scale with MLflow

Practical Deep Learning at Scale with MLflow

4.5 (11)
By: Yong Liu

Overview of this book

The book starts with an overview of the deep learning (DL) life cycle and the emerging Machine Learning Ops (MLOps) field, providing a clear picture of the four pillars of deep learning: data, model, code, and explainability and the role of MLflow in these areas. From there onward, it guides you step by step in understanding the concept of MLflow experiments and usage patterns, using MLflow as a unified framework to track DL data, code and pipelines, models, parameters, and metrics at scale. You’ll also tackle running DL pipelines in a distributed execution environment with reproducibility and provenance tracking, and tuning DL models through hyperparameter optimization (HPO) with Ray Tune, Optuna, and HyperBand. As you progress, you’ll learn how to build a multi-step DL inference pipeline with preprocessing and postprocessing steps, deploy a DL inference pipeline for production using Ray Serve and AWS SageMaker, and finally create a DL explanation as a service (EaaS) using the popular Shapley Additive Explanations (SHAP) toolbox. By the end of this book, you’ll have built the foundation and gained the hands-on experience you need to develop a DL pipeline solution from initial offline experimentation to final deployment and production, all within a reproducible and open source framework.
Table of Contents (17 chapters)
close
close
1
Section 1 - Deep Learning Challenges and MLflow Prime
4
Section 2 –
Tracking a Deep Learning Pipeline at Scale
7
Section 3 –
Running Deep Learning Pipelines at Scale
10
Section 4 –
Deploying a Deep Learning Pipeline at Scale
13
Section 5 – Deep Learning Model Explainability at Scale

Summary

In this chapter, we first reviewed the existing approaches in the MLflow APIs that could be used for implementing explainability. Two existing MLflow APIs, mlflow.shap and mlflow.evaluate, have limitations, thus cannot be used for the complex DL models and pipelines explainability scenarios we need. We then focused on two main approaches to implement SHAP explanations and explainers within the MLflow API framework: mlflow.log_artifact for logging explanations and mlflow.pyfunc.PythonModel for logging a SHAP explainer. Using the log_artifact API can allow us to log Shapley values and explanation plots into the MLflow tracking server. Using mlflow.pyfunc.PythonModel allows us to log a SHAP explainer as a MLflow pyfunc model, thus opening doors to deploy a SHAP explainer as a web service to create an EaaS endpoint. It also opens doors to use SHAP explainers through the MLflow pyfunc load_model or spark_udf API for large-scale offline batch explanation. This enables us to confidently...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Practical Deep Learning at Scale with MLflow
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon