This section will cover the different architectures for computing a continuous vector representation of words from a corpus. These representations will depend on the similarity of words, in terms of meaning. Also, there will be an introduction to a new Python library (Gensim) to do this task.
Word embeddings are a collection of techniques and methods to map words and sentences from a corpus and output them as vectors or real numbers. Word embeddings generate a representation of each word in terms of the context in which the word appears. The main task of word embeddings is to perform a dimension reduction from a space with one dimension per word to a continuous vector space.
To better understand what that means, let's have a look at an example. Imagine we have two similar sentences, such as these:
I am good.
I am great.
Now, encoding these sentences as one-hot vectors, we have something like this:
I à [1,0,0,0]
Am à [0,1,0,0]
Good à [0,0,1,0]
Great...