One powerful paradigm that we have not yet had the pleasure of discussing is the notion of transfer learning. In our excursions, we saw various methods and techniques that allow neural networks to induct powerful and accurate representations from the data they see.
Yet, what if we wanted to transfer these learned representations to other networks? This can be quite useful if we are tackling a task where not a lot of training data is available beforehand. Essentially, transfer learning seeks to leverage commonalities among different learning tasks that may share similar statistical features. Consider the following case: you are a radiologist who wants to use a Convolutional Neural Network (CNN) to classify different pulmonary diseases, using images of chest X-rays. The only problem is you only have about a hundred labeled images of...
