Using pretrained embeddings
In Chapter 9, Transfer Learning, we explored the concept of transfer learning. Here, we will revisit this concept as it relates to word embeddings. In all the models we have built up so far, we trained our word embeddings from scratch. Now, we will examine how to leverage pretrained embeddings that have been trained on massive amounts of text data, such as Word2Vec, GloVe, and FastText. Using these embeddings can be advantageous for two reasons:
- Firstly, they are already trained on a massive and diverse set of data, so they have a rich understanding of language.
- Secondly, the training process is much faster, since we will skip training our own word embeddings from scratch. Instead, we can build our models on the information packed in these embeddings, focusing on the task at hand.
It is important to note that using pretrained embeddings isn’t always the right choice. For example, if you work on niche-based text data such as medical...