Types of transfer learning
There are two main ways we can apply transfer learning in CNNs. First, we can use the pre-trained model as a feature extractor. Here, we freeze the weights of the convolutional layers to preserve the knowledge gained from the source task and add a new classifier, which is trained for classification of the second task. This works because the convolutional layers are reusable, since they only learned the low-level features such as edges, corners, and textures, which are generic and applicable in different images, as shown in Figure 9.2, while the fully connected layers are added to learn high-level details, which are used to classify different objects in a photograph.
Figure 9.2 – Transfer learning as a feature extractor
The second method of applying transfer learning is to unfreeze some layers of the pre-trained model and add a classifier model to identify the high-level features, as shown in Figure 9.3. Here, we train...