This chapter demonstrated three powerful concepts: the processing of text, Spark pipelines, and super learners.
The text processing is a powerful concept that is waiting to be largely adopted by the industry. Hence, we will go deeper into the topic in the following chapters and look at other approaches of natural language processing.
The same holds for Spark pipelines, which have become an inherent part of Spark and the core of the Spark ML package. They offer an elegant way of reusing the same concepts during training and scoring time. Hence, we would like to use the concept in the upcoming chapters as well.
Finally, with super learners, aka ensembles, you learned the basic concept of how to benefit from ensembling multiple models together with the help of a meta-learner. This offers a simple but powerful way of building strong learners, which are still simple enough...