Training and Evaluation of Advanced ML Algorithms – GPT and Autoencoders

Classical machine learning (ML) and neural networks (NNs) are very good for classical problems – prediction, classification, and recognition. As we learned in the previous chapter, training them requires a moderate amount of data, and we train them for specific tasks. However, breakthroughs in ML and artificial intelligence (AI) in the late 2010s and the beginning of 2020s were about completely different types of models – deep learning (DL), Generative Pre-Trained Transformers (GPTs), and generative AI (GenAI).

GenAI models provide two advantages – they can generate new data and they can provide us with an internal representation of the data that captures the context of the data and, to some extent, its semantics. In the previous chapters, we saw how we can use existing models for inference and generating simple pieces of text.

In this chapter, we explore how GenAI models work...