So, how can you transfer some knowledge from one model to another? Artificial neural networks have one advantage over human brains that facilitates this operation: they can be easily stored and duplicated. The expertise of a CNN is nothing but the values taken by its parameters after training—values that can easily be restored and transferred to similar networks.
Transfer learning for CNNs mostly consists of reusing the complete or partial architecture and weights of a performant network trained on a rich dataset to instantiate a new model for a different task. From this conditioned instantiation, the new model can then be fine-tuned; that is, it can be further trained on the available data for the new task/domain.
As we highlighted in the previous chapters, the first layers of a network tend to extract low-level features (such as lines, edges, or color gradients), whereas final convolutional layers react to more complex notions (such as specific shapes...
