Book Image

Engineering MLOps

By : Emmanuel Raj
Book Image

Engineering MLOps

By: Emmanuel Raj

Overview of this book

Engineering MLps presents comprehensive insights into MLOps coupled with real-world examples in Azure to help you to write programs, train robust and scalable ML models, and build ML pipelines to train and deploy models securely in production. The book begins by familiarizing you with the MLOps workflow so you can start writing programs to train ML models. Then you’ll then move on to explore options for serializing and packaging ML models post-training to deploy them to facilitate machine learning inference, model interoperability, and end-to-end model traceability. You’ll learn how to build ML pipelines, continuous integration and continuous delivery (CI/CD) pipelines, and monitor pipelines to systematically build, deploy, monitor, and govern ML solutions for businesses and industries. Finally, you’ll apply the knowledge you’ve gained to build real-world projects. By the end of this ML book, you'll have a 360-degree view of MLOps and be ready to implement MLOps in your organization.

Related Content you might be interested in

Current Title:

Small book image
Engineering MLOps
Emmanuel Raj
Apr, 2021 | 370 pages
Small book image
Azure Data Scientist Associate Certification Guide
Andreas Botsikas | Michael Hlobil
Dec, 2021 | 448 pages
Small book image
Mastering Azure Machine Learning
Christoph Körner | Marcel Alsdorf
May, 2022 | 624 pages
Small book image
Automated Machine Learning with Microsoft Azure
Mr. Dennis Michael Sawyers | Dennis Sawyers
Apr, 2021 | 340 pages
Table of Contents (18 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
1
Section 1: Framework for Building Machine Learning Models
Section 1: Framework for Building Machine Learning Models
Free Chapter
2
Chapter 1: Fundamentals of an MLOps Workflow
Chapter 1: Fundamentals of an MLOps Workflow
The evolution of infrastructure and software development
Traditional software development challenges
Trends of ML adoption in software development
Understanding MLOps
Concepts and workflow of MLOps
Summary
3
Chapter 2: Characterizing Your Machine Learning Problem
Chapter 2: Characterizing Your Machine Learning Problem
The ML solution development process
Types of ML models
Structuring your MLOps
An implementation roadmap for your solution
Procuring data, requirements, and tools
Tools and infrastructure
Discussing a real-life business problem
Summary
4
Chapter 3: Code Meets Data
Chapter 3: Code Meets Data
Business problem analysis and categorizing the problem
Setting up the resources and tools
10 principles of source code management for ML
What is good data for ML?
Data preprocessing
Data registration and versioning
Toward the ML Pipeline
Summary
5
Chapter 4: Machine Learning Pipelines
Chapter 4: Machine Learning Pipelines
Going through the basics of ML pipelines
Data ingestion and feature engineering
Machine learning training and hyperparameter optimization
Model testing and defining metrics
Model packaging
Registering models and production artifacts
Summary
6
Chapter 5: Model Evaluation and Packaging
Chapter 5: Model Evaluation and Packaging
Model evaluation and interpretability metrics
Production testing methods
Why package ML models?
How to package ML models
Inference ready models
Summary
7
Section 2: Deploying Machine Learning Models at Scale
Section 2: Deploying Machine Learning Models at Scale
8
Chapter 6: Key Principles for Deploying Your ML System
Chapter 6: Key Principles for Deploying Your ML System
ML in research versus production
Understanding the types of ML inference in production
Hands-on deployment (for the business problem)
Understanding the need for continuous integration and continuous deployment
Summary
9
Chapter 7: Building Robust CI/CD Pipelines
Chapter 7: Building Robust CI/CD Pipelines
Continuous integration, delivery, and deployment in MLOps
Setting up a CI/CD pipeline and the test environment (using Azure DevOps)
Pipeline execution and testing
Pipeline execution triggers
Summary
10
Chapter 8: APIs and Microservice Management
Chapter 8: APIs and Microservice Management
Introduction to APIs and microservices
The need for microservices for ML
Old is gold – REST API-based microservices
Hands-on implementation of serving an ML model as an API
Developing a microservice using Docker
Testing the API
Summary
11
Chapter 9: Testing and Securing Your ML Solution
Chapter 9: Testing and Securing Your ML Solution
Understanding the need for testing and securing your ML application
Testing your ML solution by design
Hands-on deployment and inference testing (a business use case)
Securing your ML solution by design
Summary
12
Chapter 10: Essentials of Production Release
Chapter 10: Essentials of Production Release
Setting up the production infrastructure
Setting up our production environment in the CI/CD pipeline
Testing our production-ready pipeline
Configuring pipeline triggers for automation
Pipeline release management
Toward continuous monitoring
Summary
13
Section 3: Monitoring Machine Learning Models in Production
Section 3: Monitoring Machine Learning Models in Production
14
Chapter 11: Key Principles for Monitoring Your ML System
Chapter 11: Key Principles for Monitoring Your ML System
Understanding the key principles of monitoring an ML system
Monitoring in the MLOps workflow
Understanding the Explainable Monitoring Framework
Enabling continuous monitoring for the service
Summary
15
Chapter 12: Model Serving and Monitoring
Chapter 12: Model Serving and Monitoring
Serving, monitoring, and maintaining models in production
Exploring different modes of serving ML models
Implementing the Explainable Monitoring framework
Governing your ML system
Summary
16
Chapter 13: Governing the ML System for Continual Learning
Chapter 13: Governing the ML System for Continual Learning
Understanding the need for continual learning
Explainable monitoring – governance
Enabling model retraining
Maintaining the CI/CD pipeline
Summary
Why subscribe?
17
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Leave a review - let other readers know what you think

Model evaluation and interpretability metrics

Acquiring data and training ML models is a good start toward creating business value. After training models, it is vital to measure the models' performance and understand why and how a model is predicting or performing in a certain way. Hence, model evaluation and interpretability are essential parts of the MLOps workflow. They enable us to understand and validate the ML models to determine the business value they will produce. As there are several types of ML models, there are numerous evaluation techniques as well.

Looking back at Chapter 2, Characterizing Your Machine Learning Problem, where we studied various types of models categorized as learning models, hybrid models, statistical models, and HITL (Human-in-the-loop) models, we will now discuss different metrics to evaluate these models. Here are some of the key model evaluation and interpretability techniques as shown in Figure 5.1. These have become standard in research...

Previous Section
End of Section 2
Next Section