Fine-Tuning and Evaluating

In this chapter, you’ll learn how to fine-tune your model on use case-specific datasets, comparing its performance to that of off-the-shelf public models. You should be able to see a quantitative and qualitative boost from your pretraining regime. You’ll dive into some examples involving language, text, and everything in between. You’ll also learn how to think about and design a human-in-the-loop evaluation system, including the same RLHF that makes ChatGPT tick! This chapter focuses on updating the trainable weights of the model. For techniques that mimic learning but don’t update the weights, such as prompt tuning and standard retrieval augmented generation, see Chapter 13 on prompt engineering.

We are going to cover the following topics in this chapter:

• Fine-tuning for language, text, and everything in between
• LLM fine-tuning breakdown – instruction fine-tuning, parameter efficient fine-tuning, and reinforcement...