Training Fully Custom ML Models with Vertex AI

In the previous chapters, we learned about training no-code (Auto-ML) as well as low-code (BQML) Machine Learning (ML) models with minimum technical expertise required. These solutions are really handy when it comes to solving common ML problems. However, sometimes the problem or data itself is so complex that it requires the development of custom Artificial Intelligence (AI) models, in most cases large deep learning-based models. Working on custom models requires a significant level of technical expertise in the fields of ML, deep learning, and AI. Sometimes, even with this expertise, it becomes really difficult to manage training and experiments of large-scale custom deep learning models due to a lack of resources, compute, and proper metadata tracking mechanisms.

To make the lives of ML developers easier, Vertex AI provides a managed environment for launching large-scale custom training jobs. Vertex AI-managed jobs let us track useful...