Summary

In this chapter, we have learned about the concepts behind distributional representations of words and its various implementations, starting from static word embeddings such as Word2Vec and GloVe.

We have then looked at improvements to the basic idea, such as subword embeddings, sentence embeddings that capture the context of the word in the sentence, as well as the use of entire language models for generating embeddings. While the language model-based embeddings are achieving state of the art results nowadays, there are still plenty of applications where more traditional approaches yield very good results, so it is important to know them all and understand the tradeoffs.

We have also looked briefly at other interesting uses of word embeddings outside the realm of natural language, where the distributional properties of other kinds of sequences are leveraged to make predictions in domains such as information retrieval and recommendation systems.

You are now ready to...