Fine-tuning
As we discussed in the first section of this chapter, the goal of model fine-tuning for LLMs is to optimize a model to generate outputs that are more specific to a task and context than the original foundation model.
The need for fine-tuning arises because pre-trained LMs are designed to model general linguistic knowledge, not specific downstream tasks. Their capabilities manifest only when adapted to applications. Fine-tuning allows pre-trained weights to be updated for target datasets and objectives. This enables knowledge transfer from the general model while customizing it for specialized tasks.
In general, there are three advantages of fine-tuning that are immediately obvious to users of these models:
- Steerability: The capability of models to follow instructions (instruction-tuning)
- Reliable output-formatting: This is important, for example, for API calls/function calling)
- Custom tone: This makes it possible to adapt the output style...
