Introduction to transfer learning

The general idea of transfer learning is to reuse the knowledge gained by a network trained for task A on another related task B. For example, if we train a network to recognize sailing boats (task A), we can use this network as a starting point to train a new model to recognize motorboats (task B). In this case, task A is called the source task and task B the target task.

Reusing a trained network as the starting point to train a new network is different from the traditional way of training networks, whereby neural networks are trained on their own for specific tasks on specific datasets. Figure 9.19 here visualizes the traditional way of network training, whereby different systems are trained for different tasks and domains:

Figure 9.19 – Traditional way of training machine learning models and neural networks

But why should we use transfer learning instead of training models in the traditional, isolated way...