9

Advanced Applications of Large Language Models

In the previous two chapters, we introduced the transformer architecture and learned about its latest large-scale incarnations, known as large language models (LLMs). We discussed them in the context of natural language processing (NLP) tasks. NLP was the original transformer application and is still the field at the forefront of LLM development today. However, the success of the architecture has led the research community to explore the application of transformers in other areas, such as computer vision.

In this chapter, we’ll focus on these areas. We’ll discuss transformers as replacements for convolutional networks (CNNs, Chapter 4) for tasks such as image classification and object detection. We’ll also learn how to use them as generative models for images instead of text, as we have done until now. We’ll also implement a model fine-tuning example – something we failed to do in Chapter 8. And...