Introduction to DL endpoint monitoring in production

We will start our chapter by describing the benefits of DL model monitoring for a deployed endpoint. Ideally, we should analyze information related to incoming data, outgoing data, model metrics, and traffic. A system that monitors the listed data can provide us with the following benefits.

Firstly, the input and output information for the model can be persisted in a data storage solution (for example, a Simple Storage Service (S3) bucket) for understanding data distributions. Detailed analysis of the incoming data and predictions can help in identifying potential improvements for the downstream process. For example, monitoring the incoming data can help us in identifying bias in model predictions. Models can be biased toward specific feature groups while handling incoming requests. This information can guide us on what we should be considering when we are training a new model for the following deployment. Another benefit comes...