The cherry on top, a cherry powered by the most sophisticated algorithms used today in the automatic construction of features for the betterment of machine learning and AI pipelines.
The previous chapter dealt with automatic feature creation using mathematical formulas, but once again, in the end, it is us, the humans, that choose the formulas and reap the benefits of them. This chapter will outline algorithms that are not in and of themselves a mathematical formula, but an architecture attempting to understand and model data in such a way that it will exploit patterns in data in order to create new data. This may sound vague at the moment, but we hope to get you excited about it!
We will focus mainly on neural algorithms that are specially designed to use a neural network design (nodes and weights). These algorithms will then impose features onto the data in such a way that can sometimes be unintelligible to humans, but extremely useful for machines. Some of the topics we'll look at are:
- Restricted Boltzmann machines
- Word2Vec/GLoVe for word embedding
Word2Vec and GLoVe are two ways of adding large dimensionality data to seemingly word tokens in the text. For example, if we look at a visual representation of the results of a Word2Vec algorithm, we might see the following:
By representing words as vectors in Euclidean space, we can achieve mathematical-esque results. In the previous example, by adding these automatically generated features we can add and subtract words by adding and subtracting their vector representations as given to us by Word2Vec. We can then generate interesting conclusions, such as king+man-woman=queen. Cool!