This section focuses almost exclusively on word vectors and how we can leverage the Gensim library to perform them.
Some of the questions we want to answer in this section include these:
- How do we use original embedding, such as GLoVe?
- How do we handle Out of Vocabulary words? (Hint: fastText)
- How do we train our own word2vec vectors on our own corpus?
- How do we train our own word2vec vectors?
- How do we train our own fastText vectors?
- How do we use similar words to compare both of the above?
First, let's get started with some simple imports, as follows:
import gensim
print(f'gensim: {gensim.__version__}')
> gensim: 3.4.0
Please ensure that your Gensim version is at least 3.4.0. This is a very popular package which is maintained and developed mostly by text processing experts over at RaRe Technologies. They use the same library in...