Implementing end-to-end language models
In the previous sections, we trained and concatenated multiple pieces to implement a final algorithm where most of the individual steps need to be trained as well. Lemmatization contains a dictionary of conversion rules. Stop words are stored in the dictionary.
Stemming needs rules for each language and word that the embedding needs to train—tf- idf and SVD are computed only on your training data but independent of each other.
This is a similar problem to the traditional computer vision approach that we will discuss in more depth in Chapter 8, Training deep neural networks on Azure, where many classic algorithms are combined into a pipeline of feature extractors and classifiers. Similar to breakthroughs of end-to-end models trained via gradient descent and backpropagation in computer vision, deep neural networks—especially sequence-to-sequence models—replaced the classical approach, a few years ago.
First, we will...