Getting online predictions from a vision model

In the previous section, we deployed our custom TF-based model to a Vertex AI Endpoint so that we could embed it into any downstream application, querying it for on-demand or online predictions. In this section, we will see how we can call this endpoint for online predictions programmatically using Python. However, the prediction requests can also be made by using a curl command and sending a JSON file with input data.

There are a few things to consider while making prediction requests; the most important part is pre-processing the input data accordingly. In the first section, when we trained our model, we did some pre-processing on our image dataset to make it compatible with the model. Similarly, while requesting the predictions, we should follow the exact same data preparation steps. Otherwise, either the model request will fail, due to an incompatible input format, or it will give bad results, due to training-serving skew. We already...