Techniques in batch model serving

In batch serving, the main steps are as follows:

1. Set up a periodic batch update of the model.
2. Store the predictions in a persistent store.
3. The web server will pull the predictions from the database.

In this section, we will go through a batch update model serving example. We will have a file, model.py, that will write some random scores for five dummy products for a hypothetical customer to a CSV file. We will set up a cron job that will run this model.py file every minute and we will fetch the customer’s data from a web server created using Flask.

Setting up a periodic batch update

We have already discussed that we can set up a periodic job using cron expressions. Usually, within a cron expression, we will run a script that will fetch the data from a database, then train a model, and then after doing the inference, will write the inference to a database:

1. As a demo of scheduling a batch job using cron expressions...