Here are some of the facts about the word2vec models that you should keep in mind when you are actually using it:
- So far, you will have realized that word2vec uses neural networks and this neural network is not a deep neural network. It only has two layers, but it works very well to find out the words similarity.
- Word2vec neural network uses a simple logistic activation function that does not use non-linear functions.
- The activation function of the hidden layer is simply linear because it directly passes its weighted sum of inputs to the next layer.
Now, we have seen almost all the major aspects of word2vec, so in the next section, we will look at the application of word2vec.