In this chapter, we examined various methods for constructing ensembles of machine learning algorithms. The main purposes of creating ensembles are these:
- Reducing the error of the elementary algorithms
- Expanding the set of possible hypotheses
- Increasing a probability of reaching the global optimum during optimizing
We saw that there are three main approaches for building ensembles: training elementary algorithms on various datasets and averaging the errors (bagging); consistently improving the results of the previous, weaker algorithms (boosting); and learning the meta-algorithm from the results of elementary algorithms (stacking). Note that the methods of building ensembles that we've covered, except stacking, require that the elementary algorithms belong to the same class, and this is one of the main requirements for ensembles. It is also believed that boosting...
