Transfer learning overview

Traditionally, a machine learning model is trained for performance on a specific task. It is only expected to work for that task and is not likely to have high performance beyond that task. Let's take the example of the problem of classifying the sentiment of IMDb movie reviews Chapter 2, Understanding Sentiment in Natural Language with BiLSTMs. The model that was trained for this particular task was optimized for performance on this task alone. A separate set of labeled data specific to a different task is required if we wish to train another model. Building another model might not be effective if there isn't enough labeled data for that task.

Transfer learning is the concept of learning a fundamental representation of the data that can be adapted to different tasks. In the case of transfer learning, a more abundantly available dataset may be used to distill knowledge and in building a new ML model for a specific task. Through the...