Transfer learning (TL)
So far, we’ve trained small models on toy datasets, where the training took no more than an hour. But if we want to work with large datasets, such as ImageNet, we will need a much bigger network that trains for a lot longer. More importantly, large datasets are not always available for the tasks we’re interested in. Keep in mind that besides obtaining the images, they have to be labeled, and this could be expensive and time-consuming. So, what does a humble engineer do when they want to solve a real ML problem with limited resources? Enter TL.
TL is the process of applying an existing trained ML model to a new, but related, problem. For example, we can take a network trained on ImageNet and repurpose it to classify grocery store items. Alternatively, we could use a driving simulator game to train an NN to drive a simulated car, and then use the network to drive a real car (but don’t try this at home!). TL is a general ML concept that applies...
