You're reading from Engineering MLOps Rapidly build, test, and manage production-ready machine learning life cycles at scale

Product type Paperback

Published in Apr 2021

Publisher Packt

ISBN-13 9781800562882

Length 370 pages

Edition 1st Edition

Tools

Azure Functions

Concepts

Machine Learning

Author (1):

Emmanuel Raj

View More author details

Table of Contents (18) Chapters

Preface

1. Section 1: Framework for Building Machine Learning Models

2. Chapter 1: Fundamentals of an MLOps Workflow FREE CHAPTER

3. Chapter 2: Characterizing Your Machine Learning Problem

4. Chapter 3: Code Meets Data

5. Chapter 4: Machine Learning Pipelines

6. Chapter 5: Model Evaluation and Packaging

7. Section 2: Deploying Machine Learning Models at Scale

8. Chapter 6: Key Principles for Deploying Your ML System

9. Chapter 7: Building Robust CI/CD Pipelines

10. Chapter 8: APIs and Microservice Management

11. Chapter 9: Testing and Securing Your ML Solution

12. Chapter 10: Essentials of Production Release

13. Section 3: Monitoring Machine Learning Models in Production

14. Chapter 11: Key Principles for Monitoring Your ML System

15. Chapter 12: Model Serving and Monitoring

16. Chapter 13: Governing the ML System for Continual Learning

17. Other Books You May Enjoy

Production testing methods

As there are various businesses in operation, so are different types of production systems serving these businesses. In this section, we look into the different types of production systems or setups commonly used and how to test them.

Batch testing

Batch testing validates your model by performing testing in an environment that is different from its training environment. Batch testing is carried out on a set of samples of data to test model inference using metrics of choice, such as accuracy, RMSE, or f1-score. Batch testing can be done in various types of computes, for example, in the cloud, or on a remote server or a test server. The model is usually served as a serialized file, and the file is loaded as an object and inferred on test data.

A/B testing

You will surely have come across A/B testing. It is often used in service design (websites, mobile apps, and so on) and for assessing marketing campaigns. For instance, it is used to evaluate...